Seeking the what and the why
Editor's note: Adrian Tennant is the co-founder and chief experience officer of Blue Kite Insight, a research agency with offices in New York and Tampa, Fla.
I recently had the pleasure of chatting with Wendy Gordon, the doyenne of qualitative research in the United Kingdom. A market research agency veteran and author of two best-selling textbooks, Gordon shared with me that throughout her career she had observed a consistent client bias toward quantitative data and a corresponding mistrust of qualitative insights.
She and I pondered a question of the zeitgeist that inspired this article: in a technology-enabled environment increasingly focused on big data, artificial intelligence tools, chatbots and machine learning, is the traditional role of qualitative research relevant? Do we need a makeover?
Tricia Wang is an ethnographic researcher, co-founder of Constellate Data and editor of the Ethnography Matters blog. Wang’s 2016 TED talk, “The human insights missing from big data”1 is a cri de coeur for qualitative research and has received over 1.1 million views. As Wang explains, although big data is a $122 billion industry, it’s best suited to quantifying environments such as electricity power grids or delivery logistics; in other words, contained systems. “When you’re quantifying, and systems are more dynamic, especially systems that involve human beings, forces are complex and unpredictable, and these are things that we don’t know how to model so well . . . relying on big data alone increases the chance that we’ll miss something, while giving us this illusion that we already know everything,” she says.
A more integrative approach
Wang’s TED talk proposes a more integrative approach to research to uncover the meanings behind big data, using what she calls “thick data” in a nod to cultural anthropologist Clifford Geertz, who frequently used the term “thick description” to explain his method of ethnography: a description that explains not just the behavior but its context as well. Big data delivers numbers; thick data delivers stories. Or put another way: big data relies on machine learning; thick data relies on human learning.
John Gambles is chairman and founder of Quadrangle Research Group in London. Writing in the Market Research Society's Market Research and Insight Yearbook, published in 2016, Gambles describes what he sees as the symbiotic relationship between research and data: “Data give us the hard numbers to put against a research-derived understanding of people and their behaviors. Data are brilliant in answering the who, what and how-much questions relating to behavior; but only research can get to the why. Research – and, particularly, qualitative research – enables us to explore and explain the motivations, expectations, attitudes, value sets and beliefs that sit behind and drive people’s behaviors; and from this, to work out how we can best impact their future behavior.”2
Where do we find an example of this integrative approach being applied? The answer might surprise you.
Online streaming entertainment service Netflix is a stereotypically quantitative, big data-driven company. In April this year, the company reported it has 104 million subscribers worldwide, 52 million of whom are in the United States.3 Netflix is known to mine its millions of subscribers’ viewing histories to predict what types of TV shows and movies people will want to watch. In an open competition, Netflix offered a prize for the best collaborative filtering algorithm to predict user ratings for content.
Netflix is not the type of company I associate with qualitative research. Yet, in a press release on its Web site, Netflix outlines how the company worked with Canadian cultural anthropologist Grant McCracken to trace the evolution of a phenomenon that streaming services like Netflix made possible: binge watching.4 Conducting ethnographic research, McCracken went into the living rooms of several TV viewers across the United States and Canada to explore their changing TV behaviors. Netflix combined the results of a quantitative survey conducted by Harris Interactive with McCracken’s qualitative insights.
Increasingly, clients’ businesses make use of big data for decision-making in the form of real-time business information and analytics, often visualized as dashboards.
As previous contributors to this magazine have noted, today’s always-on, 24/7 connected culture has accelerated the speed with which studies are designed and conducted, shortening the time available for analysis and the preparation and presentation of results.
This need for speed, combined with a pervasive technology- and data-induced attention deficit disorder, also impacts the length of client reports and the level of detail they contain. For example, at Blue Kite Insight, we typically present one- to two-page report summaries that are designed for busy executives to digest key findings in 60 seconds or less. As you would expect from researchers that measure people’s visual attention, we make extensive use of graphics and icons to communicate the most important themes, using dashboard-like layouts when it makes sense to do so. More traditional visualizations such as tables and charts are consigned to the appendices.
Are we dumbing-down because of the effects of big data? I don’t think so. The act of simplifying and summarizing has positive consequences; it forces us to consider what matters most: findings that can, in that well-worn phrase, “move the needle” for the client. Our value is in being able to surface insights that inspire clients to exploit untapped opportunities.
Deep reflection
What is under threat is the time for deep reflection; an often non-conscious process of finding the previously hidden connections between data points or the apparent incongruities that lead to unexpected insights.
A lot of our agency’s work is evaluative, pre-testing research for clients that are creating marketing communications for a significantly more fragmented media landscape than existed just a decade ago.
In the mid-1990s, I led the design and development of Web sites for international clients. Web designers routinely had to create at least two different versions of every site, each optimized for one the two dominant Web browsers of the day: Netscape and Internet Explorer. It was a royal pain for the developers but over time, the browser wars led to greater compliance with Web standards.
Fast-forward 20 years and I see parallels with digital ads and user experience testing today. Clients are being challenged to understand how people respond to commercial messages delivered through a much broader range of digital channels than ever before. While the Internet Advertising Bureau has developed standards for digital display advertisements, consumers’ expectations and behaviors differ depending on the type of online information being accessed. Think about your own use of social media compared to other types of content.
In addition to the explosion of channels, we are living in a multi-screen world. And it’s not just our domestic televisions, desktop and laptop computers, tablets, phablets and smartphones that we’re exposed to: add refrigerators and video-enabled voice-controlled devices. Digital out-of-home boards further multiply the number of messages to which we’re exposed, while quick-service restaurant (QSR) chains too are making greater use of digital screens to display animated menus and offers at the counter.
Many QSR chains now provide tabletop tablets loaded with apps for diners to browse the food and drinks menu; order; play games while they wait for their food; pay and leave feedback – experiences, it should be noted, which require only minimal interaction with a human server.
Responding to the need for faster, increasingly screen-based and geographically dispersed study requirements, our firm’s research services have evolved. For example, our online digital creative testing solution combines a survey platform with eye-tracking and facial expression analysis via respondents’ Webcams. Our mobile, quali-quant ethnographic research platform enables clients to observe customer journey maps both conceptually and geographically.
Felt by the broader industry
To get a sense of whether big data’s quantitative influence is being felt by the broader industry, I asked some of our strategic partners – the qualitative market research software companies, recruiting services and research facilities with whom we work – for their impressions.
Julia Eisenberg is vice president of iModerate, an online qualitative research firm, and I asked her if she had observed any client bias toward quantitative data. “Yes, sometimes we find that clients have a need for quantifiable qual to support their business growth in a substantive way,” Eisenberg says. “To meet this need, we’ve developed a solution that allows us to process large volumes of open-ended responses – including both stand-alone questions and open-ends tied to a survey or ratings and reviews. We use a tool that uses natural language processing to help group and categorize themes inside the data but then we deploy human intelligence to pull the story together and deliver actionable insight.”
Asked what trends in the types of qualitative client, research topics or applications have been most noticeable this year, she highlighted the use of blended technologies. “Everything from bulletin boards with immersive, ethnographic journaling and collages to in-depth text-based conversations with video sound bites for color to ongoing communities with face-to-face prototype testing sprinkled in along the way. With all of the digital and in-person tools at our disposal, we’re having fun customizing combinations that truly solve our client’s business problems.”
Steve Schlesinger is CEO of research firm Schlesinger Associates. When I asked him what trends he has observed in the qualitative side of the business throughout 2017, he didn’t divulge names but did report an increase in activity from technology-based clients, with a concentration in usability and user experience testing. He has observed the more frequent use of technology-based research methodologies such as biometrics, especially eye-tracking. Clients made more use of the bigger domestic markets such as New York City, Chicago and Los Angeles for focus groups.
He notes that more of the firm’s in-facility qualitative research came with other components, such as mobile homework assignments, or were designed as hybrid, quali-quant studies, including follow-up studies with respondents.
Some challenges
Combining qualitative with quantitative research offers the best of all possible worlds for clients but can present some methodological challenges. In a recent quantitative study for one of our clients in the travel and hospitality industry, we received 65,000 open-ended survey responses. The manual process typically used to code transcripts from qualitative depth interviews doesn’t scale well for quantitative data sets, so we employ text analytics to do some of the heavy lifting.
Our partner is text analytics software firm Ascribe and I asked Vice President Gary Zucker what has driven the adoption of text analytics as a methodology for quantitative researchers. “If pricing and timing was never an issue, you would have a human code every study, because nothing replaces the human brain,” Zucker says. “But you don’t have the time or the budget to read everything and, likely, you don’t have to read everything; you just want to find those key themes, topics and sentiments and then pull out some really detailed examples of why this is relevant. I think many brands are already there – they’re very comfortable using text analytics to help them in their business.”
Is text analytics is becoming part of the qualitative research toolkit? “I think as an industry, everybody is asking, ‘Hey, I just did a focus group. I’ve got 50-page transcripts. Each respondent is an hour-and-a-half. How do I categorize it?’ It’s hard to make heads or tails of what is and what isn’t relevant. Text analytics can help organize the data into what seems to be the most relevant, most talked about, most positive, most negative themes or phrases,” he says.
An outsized impact
This article opened with the observation from Wendy Gordon that some clients exhibit a bias toward quantitative data which devalues the importance of qualitative results. As the comments in this article from practitioners and suppliers illustrate, the idea that statistically normalized and standardized quantitative data is more useful and objective than qualitative data is flawed: small data can have an outsized impact. And qualitative research provides something that big data explicitly does not: inspiration.
So, in a world of real-time analytics and huge repositories of transaction data to mine, in what ways should researchers convey the value of qualitative methods to clients and stakeholders? What should qualitative research’s USP be? Julia Eisenberg of iModerate has a great answer: “Quantitative research tells us what but only qualitative research tells us why. Real-time analytics are valuable only up to the point where they allow for confident decision-making and we’re not always sure the numbers mean what we think they mean. We find consistent value in qualitative context – whether combined with quantitative methodologies or as a stand-alone approach. At the end of the day, numbers on a spreadsheet are not your customer. A one-dimensional understanding of the numbers can’t drive growth. Your customers – real people – are the ones who choose or choose not to buy what you’re selling. If you don’t understand them, do you really understand your business? If you’re not speaking to them qualitatively, you’re choosing not to understand their needs.”
Big data is here to stay but this presents an opportunity for researchers to become the sense-makers. Qualitative research gets behind the numbers, peeks inside people’s heads, answers the why questions and delivers the human insights missing from big data.
References
1 “The human insights missing from big data.” TED talk by Tricia Wang https://www.ted.com/talks/tricia_wang_the_human_insights_missing_from_big_data
2 “Tools for a digital world.” Essay in The Market Research and Insight Yearbook: Transforming Evidence Into Impact. Kogan Page, 2016.
3 Number of Netflix streaming subscribers worldwide from third-quarter 2011 to second-quarter 2017 (in millions), sourced from Statista.com, October 11, 2017.
https://www.statista.com/statistics/250934/quarterly-number-of-netflix-streaming-subscribers-worldwide/
4 “Netflix declares binge watching is the new normal.” PR Newswire, December 13, 2013. https://media.netflix.com/en/press-releases/netflix-declares-binge-watching-is-the-new-normal-migration-1