Scaling Up – Together: Strategies and Tactics for Scaling Your UX Research Program
Editor’s note: This article is an automated speech-to-text transcription, edited lightly for clarity.
On April 17, 2024, Key Lime Interactive and Infoblox discussed best practices for scaling up your UX research team.
Sally Cohen, director of user experience, did not have a research team when she first started at Infoblox. Key Lime Interactive helped Sally in completing research initiatives while she scaled the research team. Now, she manages a team of 12.
In this session from Quirk’s Virtual – Research Collaboration series, hear Sally and Key Lime Interactive President, Eugenio Santiago, discuss best practices for scaling UX research from their firsthand experience, with Melody Paine, UX research manager at Key Lime Interactive, facilitating the discussion.
Session transcript
Joe Rydholm:
Hi everybody and welcome to our session “Scaling Up – Together: Strategies and Tactics for Scaling Your UX Research Program.”
I’m Quirk’s Editor, Joe Rydholm, and before we get started lets quickly go over the ways you can participate in today’s discussion. You can use the chat tab to interact with other attendees during the session. And you can use the Q&A tab to submit questions for the presenters during the session and they will answer as many questions as they have time for during the Q&A portion.
Our session today is presented by Key Lime. Enjoy the presentation.
Melody Paine:
Hi everyone. I'm Melody Paine, user experience research manager and lead researcher at Key Lime Interactive. And I'd like to welcome you to our virtual session, “Scaling Up Together with Strategies and Tactics for Scaling Your Research program.”
I have two guests here with me today. The first is Sally Cohen, director of user experience at Infoblox. Sally, would you like to introduce yourself?
Sally Cohen:
Yeah, hi everyone.
As Melody said, I'm Sally Cohen. I work for a company called Infoblox, which does networking and cybersecurity. I manage a UX team of 12 right now, and it includes both design and research. I have a long history of managing primarily enterprise teams that do both design and research and have built-in and scaled several teams.
Melody Paine:
Welcome. And next up is Eugenio Santiago who goes by Eugene. Would you like to say a few words about yourself, Eugene?
Eugenio (Eugene) Santiago:
Sure, thanks, Melody.
I'm currently the President at Key Lime Interactive. I'm a UX researcher by trade, and I've worked my way up the ladder from support to lead UXR. Then I was responsible for growing and scaling a team and responsible for our collective quality.
And today I manage the entire research business at Key Lime. We're a full service UX and CX research agency. We run projects end to end and we help our clients, like Sally, scale their research practices.
Melody Paine:
Great. Okay. Well, with that in mind, we are going to get started by discussing these topics today.
The theme of the event is research collaboration and scaling up together. So, we're going to touch on team empowerment, strategic planning and technology integration with some of the questions we're going to get into.
So, let's talk about getting started with scaling up your practice. What is one of the first steps you can take to scale your internal practice? Let's start with you, Sally.
Sally Cohen:
Sure. So, I think before you can even think about scaling a practice, you have to make sure that the organization has buy-in to conducting user research.
I would say over the years, the need to justify research has greatly diminished, thank goodness. I think years ago I could speak to people who didn't understand what user research was and why it was necessary. I think now most stakeholders get the importance of doing research, but understanding why do you need to execute it on a regular basis? Why do you need to have resources that are embedded within your organization to help you do it is really important.
So, I think the first thing is to ensure that there's buy-in. Sometimes what it takes to do that, however, is to just start doing it. And if you don't have internal resources, I think that's also a good opportunity to bring in some extra support. I know here at Infoblox when I started, I did not have a research team, and so that's when I called Key Lime.
It's important that you show some wins in order to get people to understand the value and really start to invest in it at scale.
Melody Paine:
Okay. Anything to add to that, Eugene?
Eugenio (Eugene) Santiago:
I mean, we've had an opportunity to work with Sally for a number of years through her UX career. So when she called, we were excited and happy that a new opportunity had come up. And having done this with Sally in the past, we were in a position where we could offer her a mix of resources to sort of fulfill her needs.
Like she mentioned, she was getting started, she was getting acclimated. She has a long road ahead of her in terms of building that momentum. But like she mentioned, above all, she needed to start stacking the wins. And so, we helped her with someone who could help run some of that tactical research pretty quickly and help her do that. But then there were other folks on the team who could help be a peer to Sally and kind of work through some of those longer-term initiatives.
And so I think when it comes to sort of setting the foundation and building a team, if you can have folks who can assist on both the short term or tactical needs and help you chip in on the strategic initiatives, then I think it helps you build a good cadence.
Melody Paine:
Great. That's a good segue into team empowerment.
So, how do you maintain successful partnerships internally and with resources? So, let's start with you, Eugene, your thoughts about that.
Eugenio (Eugene) Santiago:
Yeah, I mean, so for us as a research vendor, and I think about our most successful engagements, it's really when our researchers take an increased degree of accountability and ownership.
This notion of partnership is really important to us. And so, whether it's my team or other researchers that I mentor in the industry, I encourage them as researchers, tap into your empathy skill and apply it to your stakeholders. So, think about how they measure success.
So, this is what I thought about. How does Sally measure success? What challenges is she facing? We weren't just focused on the research question. And that for me, it opens up an opportunity where you think about how research can empower the organization and you're not only focusing on what you can learn from research. And if you're able to do this effectively, I feel it really creates a collaborative team environment, which I think is a critical part of sustaining relationships.
Melody Paine:
And then Sally, what about internally? Some of those relationships and partnerships?
Sally Cohen:
Yeah, so we spent a lot of time talking about our stakeholders. What does that mean to have a stakeholder?
I think part of what user research brings to the table is building empathy to your end users for your end users in the organization as a whole. And so really everybody that affects your product should be one of your stakeholders. And certainly from a UX perspective, arguably our most significant stakeholder is product management.
And I think what's really important is that product management doesn't just say, ‘okay, go off and do research. That's lovely. You guys learn what you want to do and let us know,” but they're actually an active part of the experience and that they're not only bought into you executing it, but they want to execute it with you. They want to be part of that process. They want to understand the learnings with you, and they grow that understanding together.
And so, we see ourselves not just as, ‘oh, we're accumulating all this knowledge and our knowledge is our power.’ It's really about bringing that knowledge into the organization and we're just the conduit to help make that happen. And I think that's what makes us really effective in the longer term because yes, it is nice to have information other people don't have, but ultimately you've got to be an effective partner to the rest of the organization and to help build the knowledge within your whole company and not just within your team.
Eugenio (Eugene) Santiago:
I mean, I certainly agree. And that's what, again, I preach to every and any UX researcher that I can.
What Sally mentioned, tie it back to the business goals. Again, understand what the KPIs are, like you said, it's not information for the sake of information. It's information to empower action, to build confidence. And you need to do that hard and persistent work of understanding what is it, why are you even doing the research? What business decision are you hoping that you're able to unlock? And the more that you do that, then I think it helps you get to where you want to go.
You do become the question answerer. You're like the insights whisperer where the shift happens, where people see you as a conduit for helping them be successful, helping the product be successful and then the floodgates open. Right now you've got a backlog, right, of work and those kinds of things. It's important. It's persistent. You need to keep doing it, keep honing it.
Like you said, like Sally said, when it works, it's great. It's a true team effort. And the maturity from UX implementation starts to, starts to really elevate, and that's when it gets really fun.
Sally Cohen:
And I want to piggyback on something Eugene said that I think is important is not just about the research but about the business goals.
So, what I think too often we do in UX is we kind of geek out on our own little understanding of things and our findings and our own little minutia, which is cool, but at the end of the day, all of this has to transfer and translate to the business. And if we don't understand why we're doing what we're doing and ultimately how this is going to affect sales and revenue and renewals and all of the things that the business gets measured on, then ultimately our research falls a little flat.
And so it's important to get our heads above the water a little bit and not just understand, ‘gee, what is this end user doing? And here's the tasks they perform and how they perform, but here's why it is relevant to the design, to the organization, to our business.’ And that is a different level of maturity that you have to build within your practice to get to that point. But that is a sea change. When you start to speak business speak, that's when I think significant buy-in happens and you get more attention and more resources.
Melody Paine:
Great. Great thoughts.
I've heard you talking a little bit about building the appetite at the organization and then also you mentioned confidence and getting your practitioners to be more confident. What's your approach? Maybe you both have something to say about this, but Sally, what's your approach to coaching and learning and development on the team?
Sally Cohen:
Yeah, I think, and it depends on the type of business you have, but in most organizations, the domain in which you work is so critical and understanding that.
I work in a business where we do networking and cybersecurity, a very technical domain. None of us come from a background where we were managing a company's network. That's not the background that we have. But we have to be able to know enough about that domain to have enough understanding and empathy about what the end users are doing. And ultimately helping the researchers understand and be able to speak the language of the people that they're speaking with is really an important part of the coaching.
I'm sure in many organizations that is critically important. It's not just enough to be a good researcher in the understanding of how to practice the techniques, but also how to truly be a good communicator with the end user so that you can understand and translate what they are saying. So, I think that is really critical.
I also think it's really critical to work on communication skills. I have this little mantra that I say that in both research and design, 20% is actually doing the work and 80% communicating about the work.
The communication piece of it is so critical. You can do the best research in the world, but if you don't have a good way of translating that to both your designers as well as your stakeholders, like product management or engineering, it doesn't matter how good your research was, nobody will buy into it. And they start doing things like questioning, ‘well, you didn't talk to the right people or you didn't ask the right questions.’
So being able to translate that into something that people can understand and embrace is a significant part of the job. And honestly, that takes so much practice. You've got to give people those opportunities to work on those skills. If your researchers are simply behind the scenes just doing analysis, but they're not the ones communicating, give them those opportunities to communicate because that's where their skills will build, their confidence will build and they will shine within the organization.
Eugenio (Eugene) Santiago:
I'm going to steal that 80-20 because I believe it too.
Sally Cohen:
Yeah, absolutely everybody likes that one.
Eugenio (Eugene) Santiago:
No, it is. I think ultimately, right, the name of the game, like you said, it doesn't matter. You can do the best research, but you could do it in the dark if it is not relevant, if it doesn't resonate, if you don't know how to translate it into the language of your stakeholders so that they're receptive to it, so that it initiates action for them.
Melody, if I can, for us, the challenge is a little bit different in terms of training and scaling. When I think about how we support Sally, it really puts our research ops to the test.
So, a lot of things that we do are things like when we have our first two people setting the groundwork, understanding Sally's needs, her organization's needs, the language that they speak, starting to build some of that continuity, we really try hard to operationalize that.
So, we create client playbooks. For us, the goal is to try to take on some of that responsibility that Sally has when she onboards someone say each time and you feel like, ‘oh man, you have to start from scratch.’
We've gotten pretty good at identifying ways to feed people enough information so they can get started, but then sort of provide an apparatus of support around them because you don't become an expert in network admin overnight, but there's always that balance of we got to get the work done, we got to get started and we have to sort of acclimate the team and stuff like that.
So those are some of the things that we do. And something to think about even if you're building a team internally, is how do you set them up for success?
Like Sally said, there's the research part of it, but then there's the business knowledge. And so, balancing those two and getting them prepared on both fronts goes a long, long way.
Melody Paine:
Great. Okay. Well, we have a couple more topics I'd love to hear your thoughts about. And one is strategic planning.
There's discussions of things called research roadmaps or just annual plans, those kinds of things. Sally, would you like to get us started by talking about creating those in the first place? In case there's anyone listening here who hasn't gone through that yet?
Sally Cohen:
I think every business has this annual planning cycle that they go through. And I do believe it's important for any UX team in concert with that annual planning, due planning around what's the UX team going to do and how do we best support the company's initiatives?
And doing that on an annual basis makes great sense. So, what are the main initiatives? And really understand what we as an organization know. What do we assume and where do we think we need to grow in our understanding? And I think having that sort of outlined and being able to then establish research around those things is really important.
But what I make sure I don't do is say, ‘here's my priorities for the year,’ and then I just stick to it like glue. Because we all know, I think most of us are working in agile development environments, right? And even I think before we were agile, priorities change. Some new idea comes in and it throws you for a curve ball and you've got to react to it.
And research is no different. Even if we're doing generative research, which has a long tail, it's a long process. You still need to adapt and change and adjust to what the new expectations are of the organization.
So, on a quarterly basis, we're constantly going through and saying, ‘alright, what have we done to date? What do we need to keep doing? What should we stop doing because it's no longer a priority and what do we need to add to the mix and put that in our plan?’
It's really a fluid process. It is not one where you have a plan and you stick to it. It inevitably changes. I think the most successful teams are the ones that can adapt to those changing priorities and say, ‘you know what? Stakeholders, we got your back. We can adjust and we can help learn when we need to learn something new.’
Melody Paine:
Great. And how does that impact the relationship if you do have outsourced researchers keeping up with your plans, the fluid changes, as you said?
Sally Cohen:
Yeah. Yeah. And I think this is something Eugene has talked about before, when we've had Key Lime researchers on our team. They're a member of the team, which is really helpful so that they're part of the planning.
We are communicating as a team what are the things that are coming up and what are the changes and what's the next thing we're going to be working on so that they're part of the regular routines that we're having. And it really helps to make that communication a little more seamless with any resource, whether they're internal or external.
It's a little challenging. People can get whiplash when priorities change too much and they're like, ‘whoa, what just happened to that?’ But I do think when you have regular routines where you are constantly reviewing your roadmap and the plan, it's helpful and people can anticipate, which is also really helpful.
When we've had outside resources, they can say, ‘all right, I'm going to be done with this in a couple of weeks. What's the next thing you want me to do?’ So, we can start to think about it in advance. That is always really helpful as well.
Melody Paine:
Did anything come to mind for you, Eugene, in terms of advising organizations about what to think about or what questions to ask themselves when they're doing this, kind of planning out their research for the future?
Eugenio (Eugene) Santiago:
Yeah, I mean, yes, I certainly agree with Sally in the macro level approach and us as researchers, I think a good chunk of us, we like to plan, we like to organize, we like to execute, said plan. But like Sally said, one of the constants is that things will always change. Priorities always change. And so, as researchers, as you get more acclimated in the field, part of the learning part of maybe a hard skill and a soft skill is finding that balance between research rigor and what the business needs are.
And it's a labor of love. It's difficult for many of us, but it's the type of thing that comes with the territory that you have to get better at. For us internally, we call it being dynamic. It's essential. We think it's essential to being a reliable resource, someone who can fill the gaps as they arise.
It takes practice though. So, I would say for anyone listening, maybe finding themselves facing that challenge, keep at it. You're going to get better at it, but you have to get better at it, right? Because it goes back to what Sally mentioned, we're service providers. We're constantly needing to show utility and being helpful, and we want to build trust. We want to build confidence. We want people to keep coming back to us and asking for our opinion, for our support. And it's the nature of things.
So, I would keep encouraging people. It's difficult, and it doesn't happen overnight, but it pays off big time.
Melody Paine:
Great.
So, it sounds like planning is a big part of the process and reassessing the plan.
So, let's talk a little bit more about technology and process integration. And as you're getting your research practice growing and even off the ground, how do you know what to change and what to keep the same about your research process when you're first looking at it? I'll start with you, Sally.
Sally Cohen:
Yeah, I would say when I started, as I mentioned, I didn't have any researchers, so it was really just executing something and showing some wins. What's the value of having some insights that we didn't have before? And that got me to the point where then I could add a researcher, and then I had the one lone researcher on the team.
The way I looked at it was, ‘okay, I could spread him really thin and have him try to do everything for everyone, or we could try to focus him and be successful in a key area where we haven't had any attention.’
So that was really the point I was making. Especially because I did have some support still from Key Lime that I could focus the Key Lime resources in one area and have our new person focus in another area and give that some care and feeding without trying to spread everybody across too many things and trying to learn too much and be all things to all people at one time.
And so that was another point of reassessment in terms of how do I do that? And then I added more people, so how do I attack it at that point?
So, as you justify an increase in resources, I think obviously it always means you should reimagine how your practice needs to work. And you start to think about, ‘okay, what is something that an individual does versus something that you perhaps have a team to do?’
You also need to think about people at different skill levels. If you bring in senior researchers and you bring in junior researchers, what does that look like? So, you have a mix of skills within the team, and you have people who may be doing more generative research and some who are more evaluative research. You have to consider all of those factors as you are scaling your team.
I don't know that there's a cookbook for how to do it, but I do think you have to think about those decisions. How do you help people grow and learn both within their new job and in their career and decide how you're going to allocate resources appropriately?
And then you talked about technology, and so I think one of the things that's also important to consider as you're growing the team is when do you start to invest in the tools? I mean, there's a myriad of tools out there for research nowadays. I've never thought about it as scaling a research practice is this big bang where I've got to buy everything all at once. I've got to bring in all the resources all at once. I don't think that's the right approach. I think about tools when I need to think about how I can operate at scale.
So I don't have researchers reinventing the wheel. I don't have them creating a new screener every time they need to recruit. I don't have them creating a new interview protocol every time they interview. So, are there tools that I can use to enable us to do that more successfully so that we're not being repetitive?
And for me, I think the biggest needs we have are to be able to do both evaluative research at scale and to do recruiting at scale. And there's lots of tools that enable us to do that more successfully. Right? That's when I said I need to invest in tools because recruiting can be a very difficult and expensive thing. And for any of you who work in an enterprise, know that getting to your customers can often be challenging. So, having the tools to be able to recruit successfully have been very important to us to be able to utilize the researchers that we have.
So that's been an important factor for us. But I wouldn't have tried to do that with my researcher of one. It wouldn't have made sense.
Ultimately, as I've added a team, it's been a wise investment to help us be more efficient and effective as we've had a team.
Melody Paine:
Great.
Eugene, how do you think about process technology tools?
Eugenio (Eugene) Santiago:
I agree. I'm aligned with Sally in that we view tools as process enhancers.
It's interesting. Tools certainly can help you scale, but I would agree with Sally. We kind of back our way into it in terms of not necessarily thinking, ‘oh, the tool is going to help us scale.’
For us, it starts before the tools. Sally alluded to this a little bit in terms of not recreating the wheel, the more deliberate you can be about creating a repeatable process for research, project execution, then I think you're doing a really good job of setting an appropriate foundation that manifests itself in lots of different ways. When you start taking in requests. Simple things like just making sure that you are collecting those research requests in a formulaic way goes a long way into enabling you to triage requests as they come in, in an objective way.
To do that, that takes, like Sally mentioned, in terms of building internal relationships, educating your peers, educating product teams, product owners, what information do you need? If you can show them how to send requests in sort of a cookie cutter way, or it has the things that you need, you can move through the assessment process and ultimately the execution process of research a lot more smoothly.
Another thing that is important as researchers, when we think about operations, like Sally mentioned this communication, communication, communication, build that into the process review and get confirmation that your understanding of the needs are what your stakeholders needs. It goes a long way into helping manage expectations.
Again, you'd be surprised if you start creating and building these habits. The scaling can happen, the addition of people to your way of your running research is the beginning of starting to scale.
And then once you have that established, then for us, we certainly look at tools to do exactly what Sally said, how can we be more efficient? How can we be more effective? How can we continue to scale?
Inevitably, even when you start to build the momentum, business gets in the way to a certain extent. So, you're constantly trying to figure out how can I effectively get more with what I have? And that, I think, for us is where tools really start to open a piece.
And to Sally's point, research for the most part, has a very specific cadence, sort of independent of the questions that you're asking for. You go through a consistent journey of steps, study, design and scoping. The logistics and the research optimization part, the data collection piece, tools can't necessarily, at least today, they can't help you through all of it. But for the laborious parts, like the recruiting part, sometimes depending on the type of information you need, the data collection piece, and now with AI increasingly more on some of the synthesis and analysis pieces, those are the parts where I think tools can really help you grow your efficiency at an exponential rate.
But you got to have a well thought out process, right? Something that's repeatable, because then you can be really effective at choosing the tools that really help you the most. For some people, like for Sally and for other researchers, the recruiting stuff, important but laborious, the data collection part, laborious, time consuming, really fun sometimes, but you have to pick and choose where you put your time.
A mantra that my wife has in our family, both at home and at work, is constantly assessing and identifying what's the best use of my time. As researchers, when you get really good, you can do everything end-to-end. The challenge becomes where's the most impact? And tools that allow you to spend more of your time, like Sally said, on that 80% of the crafting the story, the understanding it, tying it back and less on the research part in terms of the execution part, the part that we see and we breed every single day. And so, you start to kind of define yourself by that. That's when tools start to unlock a completely different sort of level of impact.
So, that's how we approach it.
Melody Paine:
Great summary.
So, to wrap things up here, I'd like to ask you both if you have any advice for folks who are listening on how best to remain collaborative while scaling up? Are there a few key things that you'd like to have people keep in mind as we wrap up here? Sally.
Sally Cohen:
One of the things that I often get asked is what's important to me when I am hiring? And I'm reflecting on that now, that the word that always comes to mind to me is curiosity. And I think when people are curious, that breeds collaboration.
I think collaboration has become this buzzword that everybody uses, and everybody thinks they're collaborative because they have a conversation. But I think the key to really being collaborative is to communicate with curiosity, because then you're not just there to talk. I mean, maybe I am right now. I'm here to talk, but ultimately you're listening. You're not just sharing, but you're also listening to what others say.
I do believe that when you have that curious spirit, you are much more effective at working with others. I think that is, of course, I talk about this with respect to researchers and the need to have curiosity in their core, but I think it's true for everybody. And I think that is ultimately how collaboration spreads within the organization, is that nobody assumes they have all the magic answers, and they remain curious about what others think and feel and believe. And I do think that's what makes a difference in how you be effective in working with others who have maybe different points of view and different perspectives.
Melody Paine:
Eugene, any key things to leave for folks to keep in mind?
Eugenio (Eugene) Santiago:
To piggyback on what Sally just said about the concept of collaboration, when I think of collaboration, like really good collaboration, the type that Sally was describing, another sort of buzzword comes to my mind. It's alignment, like true teamwork, traction towards a shared and common goal is I think another sort of foundational piece.
So, just to bring it back full circle to some of the stuff that Sally said at the beginning, as a takeaway, tie your research findings back to the business value. Interview your stakeholders. Figure out what success looks like for them so you can figure out how you can support that success. Again, that goes a long, long way.
We're researchers, we're rigorous about what we do. We're planners. Remember, stay flexible because things are going to change.
And find the balance between wanting to do what we, as researchers might inevitably feel is more impactful, like strategic stuff and focusing on doing the strategic stuff. Never downplay the importance of stacking wins. You need to do both. But if you want to do more strategic stuff, you got to stack those wins all the time. The tactical stuff, building that momentum, building that to an extent, that degree of dependency. Be the person that gets the answers that moves the ball down the field. Even if it's just a little bit, they'll keep on coming back.
Then you can do the longer term, more strategic things, the things that inevitably we might get a lot more joy in and feel like it's more impactful, but stacking those wins, it's important. It's really important.
Melody Paine:
Okay, great. I think that wraps up our time together, so let's take 10 to 15 minutes to answer some of the questions that are coming in live from the audience.