Editor’s note: Emily Howard is senior research manager, and Saara Rietmulder is research business strategist, at Microsoft Research + Insights, Redmond, Wash.
Microsoft’s Research + Insights team has been on a long journey toward research innovation, focusing on digital transformation. This effort has allowed the team to use pioneering techniques, connecting survey and behavioral data to better understand market mix and revenue growth opportunities, and increasing the use of other sources of data – like telemetry and social – for richer insights.
Prior to COVID-19, our team set its sights on a different space: qualitative research. We wanted to break out of the repetitive in-depth, one-to-one in-person interview and focus group molds and think outside the box.
Disrupting the paradigm pre-COVID-19
Not long ago, the biggest trend in the industry was acquiring enough sample to complete better and more sophisticated modeling. Microsoft’s Research + Insights team created complex driver models, did analysis comparing behavioral and perception analysis, and predicted changes to purchase intent. Research capabilities exploded with these new techniques in data science and digital transformation made the analysis possible.
While valuable, the reliance on quantitative sample left researchers wanting more depth. In addition, the Research + Insights team felt that traditional qual methods were starting to feel clinical and unnatural.
Working to refresh the traditional approaches, our team focused on co-creation sessions, bringing customers and product managers together to jointly solve problems and discover insights. Meeting rooms, round tables and one-way mirrors were replaced by shared spaces with couches and comfortable seating. Respondents were more relaxed and stakeholders were directly engaged. While these approaches didn’t work for every project, the disruption to the paradigm was infectious.
Then, COVID-19 sent everyone home.
"When we were still doing in person qual we were pushing to bring in a lot more creative approaches, something beyond those behind-the-glass clinical groups! We have done focus groups in “lounge” settings where the environment feels a bit more personal, we have done groups where we set up a salad bar and equated putting together a salad as a parallel to choosing a cloud provider. Lettuce as the foundation is equivalent of security when it comes to a cloud provider. Ever since we have switched completely to online qual, I started to feel like we were taking a step back to the more clinical approach without innovation." – Director of market research, Microsoft
To keep moving forward, our team had to re-think some of these traditional in-person methods with an eye to digital native experiences.
COVID-19 forces innovative methods
When COVID-19 hit, Research + Insights’ first reaction was to go back to the old standard of individual interviews and focus groups, just via the phone. While in most instances this worked, the tools felt clunky and dated. The team started asking, “What does a digital-first qualitative experience actually look like? What are the current trends? How can they do something new? And most importantly, how can it bring value to Microsoft?”
Stepping back, we realized that most people experience a lot of their digital lives in text-form. The pandemic didn’t change that but did force us to all to incorporate video calling in our daily lives. Our team felt we were missing options and began looking for methods that were closer to people’s existing online experience.
A team was formed with members from the commercial and consumer sides of the research team. They fielded an RFI to more than 35 research companies to look at the tools being used across the industry, narrowing it down to eight that seemed to be thinking about digital-first research design. After reviewing new cloud-based platforms and learnings across key research companies, the Research + Insights team established some best practices for traditional approaches in a digital world and moved away from approaches that simply weren’t working. We then held sessions with the team of 80+ researchers to discuss the trends and methods in the industry and the digital-first experience that could take the team through the next phase.
Key takeaways
Cloud-based research tools
Cloud-based research tools have improved in user interface experience, functionality and analysis capabilities. The tools more closely mirror our natural online experiences, resembling social media more than clinical research. New features have been added beyond text and polling, expanding methodologies that are available and allow us to get more creative. Research + Insights looked at both synchronous and asynchronous methods, allowing us to create curated task flows or pivot to real-time conversations.
These platforms allowed our team to complete qualitative research at a scale we haven’t been able to do before, blurring the qual/quant divide of years past. Now, data-driven results could be mixed with storytelling in a matter of days instead of weeks. All of this was made possible by technological improvements and gains in natural language processing, real-time translation and text processing.
"In this remote working world, online board methodologies have been a nice addition to traditional in person approaches. The online methodologies have allowed us to simulate experiences that are closer to what we are testing, like website copy, where there are no other people or a moderator to explain something or react to. The mixture of ability to get structured and unstructured feedback means we can get a lot of insight in one stage faster compared to traditional qual and quant stages. We will absolutely use them more moving forward." – Director of Microsoft Azure product marketing
These tools also gave them more flexibility in respondents. Global recruiting was more accessible, as it was not limited to a physical location. The digital experience allowed for the same group to engage over a period of days or weeks to test longitudinally. Added flexibility meant our team could focus on the right audience and be pickier with respondent selections. Meeting respondents where they were – and not always having to commit to a certain hour in a specific location – meant researchers were having higher recruit success rates.
Cloud-based tools brought additional benefits, including: not worrying about one respondent dominating the conversation and coloring others’ response; the ability to change discussion guides mid-research; and easier, more robust analysis of results with robust segmentation.
“Since the start of COVID-19 – and accelerated by it – Hypothesis has implemented a thoughtful, digital-forward approach to qualitative research. Coming from a creative, objectives-first mind-set, we pivoted the way we structure research – mixed methodologies, shorter sessions – how we facilitate the conversation – high quality online platforms, virtual flipcharts, interactive stimuli – and how we craft deliverables – unique imagery, respondent provided B-roll – to produce actionable insights and dynamic reports, videos and more.” – Shaina Green, senior director, Hypothesis
Focus groups and interviews
Focus groups and interviews are a tried-and-true method, but they are a little different when performed online. The clearest positive is that respondents are increasingly more comfortable in online video settings and using video-based platforms. People are more relaxed in the conversation and familiar with the functionality. On the other side, it is difficult to engage a traditional six-to-10 person group in a conversation online and researchers need to manage the conversations more directly.
The Research + Insights team started limiting online focus groups to four or five people and coached moderators to be more directive in the conversation. When conducting focus groups online, natural conversation flow is a little awkward as people do not pick up cues as easily. Moderator must call on people for a response. This reduces interruptions, makes it less likely for respondents to talk over each other and helps avoid the “sorry, you go first’s.” Respondents are more engaged when they are provided with some stimulus to react to or mentally focus on. While this is true in person, the effects seem to be more impactful in the online experience.
Though we misses the more casual, in-person sessions, our team still gets quality insights using these online formats. Groups, along with one-on-one interviews, still make up a large portion of the Research + Insights research portfolio.
Creating a digital-first experience
Some methodologies simply don’t work as well in an online-only setting, specifically ethnographies and ideation.
While there are new tools and features that have significantly improved these remote versions of these methods, there are significant gaps in the ability to gain meaningful insights. There has been some success in these methods, when a lot of up-front design coupled with pre-session homework is implemented. For now, we recommend shifting from these methods to other digital-first options until in-person research is viable.
What will happen post-COVID-19?
While no one can predict what will happen next, we can make some assumptions. Once we can gather in person again, many interviews and focus groups will come back, as will the ethnographies and ideation sessions.
However, online research has brought an unprecedented speed in netting results that will dominate moving forward. The rule of thumb will likely be to continue using online methodologies unless it is proven it needs to be done in person, not the other way around. This will likely be the new standard across the tech industry. In-person focus groups will likely occur at conferences where people are already gathering, lessoning the individual trips of the past.
“Modern research was already experiencing a shift to digital. COVID-19 accelerated this transformation creating new opportunities to embrace the digital-first experiences customers are actively engaged in today, and to evolve our measurement tools and technologies to explore human engagement in entirely new ways.” – Tammy Kaneshige, partner, Emerald Research
It’s worth noting that many tools were not new as of March 2020, rather their use has changed, thrusting them into the forefront. Researchers are more comfortable relying on online methods because the world is now more comfortable in – and reliant on – these spaces. Innovations like Microsoft Teams and other streaming functions have further propelled the capabilities, which will only improve with time.
Looking forward, researchers at Microsoft will continue to focus on how to best create a digital-first experience when conducting online research.