More data, less hassle

Editor's note: Based in Denver, Julia Eisenberg is vice president of insights at research company 20|20. Based in Nashville, Tenn., Isaac Rogers is CEO of 20|20.

Some great ideas are born at the wrong time, before the world is truly ready for them. Webvan, the home grocery delivery service, went bankrupt in 2001 in spectacular dot-com fashion. Microsoft launched its SPOT smartwatches way back in 2004, only to close the program four years later.

Yet, when you look down at many people’s wrists today, you’ll find some version of an Apple Watch or Fitbit worn proudly. Most of you reading this article have likely ordered some sort of at-home delivery service from Instacart, Amazon Prime Now or Uber Eats. When you look at these examples, it’s puzzling why sometimes it takes so long for society to catch up to technology or ideas that were readily available a decade before we all realized the benefits. 

In many cases, these before-their-time failures happened because the ideas were so radical we didn’t really believe or trust they were possible. In some cases, there needed to be some iterative steps in the path between the existing way of doing business and the future state; we almost had to learn A followed by B before we could ever conceive of C, whereas these too-early disruptive concepts jumped too far and too fast for us to keep up.

In the marketing research industry, one such methodology that was simply launched before its time is now seeing a resurgence as marketers and brands increasingly reap the benefits: hybrid quant+qual research for advertising, message and concept testing. 

Well over a decade ago, researchers first started experimenting with ways to integrate qualitative insights into their quantitative research. The promise was simple and obvious; the approach was a way to bridge the quantitative testing data with deeper understanding from the respondent through some type of digital qualitative interview, video conversation or online group discussion. A marketer would then have a robust set of projectable data and a winning set of concepts. Through the qualitative insights they would understand why these concepts succeeded when others failed, perhaps even be able to take a “good” winning concept to an even higher level and be better equipped to develop the next iteration of their brand or product story. 

However, it took practitioners a while to figure out the best ways to get the most value out of the increased effort. In some cases, the technology to integrate the hybrid chat or video intercept wasn’t very reliable or simple. In other cases, panel providers were wary of letting their valuable panelists have direct, one-to-one conversations with market researchers. These early friction points proved fatal for the widespread adoption of hybrid insights across our industry. So, just like Webvan and the SPOT watch before it, hybrid methodologies for ad, message and concept testing seemed to recede into the background, waiting for the right moment to reappear. 

Forced to adapt

So why are these approaches now experiencing a resurgence? Developing a top-performing ad campaign or launching a new product to market has never been as hard as it is now. In the past few years, marketers have been forced to adapt to a world where consumers are exposed to hundreds of ads from competing brands and products every day. Digital media radically increases the exposure of consumers to advertising content and makes it incredibly difficult to stand out among the background noise. Years ago the old rules of share-of-voice and ad exposure began to shift dramatically, making marketers’ jobs much more difficult and the consumer that much harder to reach. 

For this reason, many marketers and insights teams are realizing that their old instruments for testing and optimizing creative won’t be competitive going forward. It’s not enough to simply find the best ad version out of your top seven; your ad now has to be the best version it can possibly be, hitting every mark with the consumer in the hopes of standing out in increasingly crowded spaces. Consumers are more savvy about marketing messages and better at tuning them out. Not only do ads have to be fully optimized but it becomes more challenging every day to find ways to connect with consumers’ ever-evolving needs and preferences to deliver a compelling and authentic message. 

Reenter, hybrid quant+qual research. Over the last couple of years we’ve seen a massive increase in the ways marketers and insights teams are leveraging advanced qualitative methods within their existing quantitative ad testing regimen. The marketer has evolved enough to accept that more is more – more information to make faster, more confident decisions with more context about their work is crucial. Additionally, the technology is finally optimized for clear, direct, turnkey ease of use. In the past, taking a risk on a hybrid evaluation method meant potential delays to deadlines and headaches over complex, buggy tech. Today, the tools and integration are seamless. The only risk is forgoing a qual+quant assessment and missing something.

Nothing stood out

Let’s take a look at a few examples of the hybrid approach in action. 

A global cosmetics client was deeply frustrated that none of the seemingly dozens of versions of its most recent display ad seemed to resonate. Sure, there were a few treatments that benchmarked slightly higher than the rest of the pack, but nothing stood out and the firm was not willing to fully launch the campaign until it had the perfect message and visuals to accompany its new product. Through rounds of testing, it kept weeding through ideas only to find itself without a clear, breakthrough contender. It decided there was something missing – the why. Why were some ads testing slightly higher? What nuances could the firm focus on, or what language could it highlight, to fully optimize the best version of its work? 

To uncover the why, it turned to a hybrid approach that asked a portion of its respondent audience to join it in a brief two-day online discussion. Forty women were randomly selected to participate in this quick, online community where the brand’s qualitative team engaged them in a deeper, more thoughtful conversation about three of the top-performing ads. 

While there were subtle improvements that could be made in all three treatments, one version kept being referred to as “awkward” or “unusual.” With a little bit of digging, the researcher discovered that nearly every woman in the target audience found a single phrase in the copy to be off-putting. The phrase just didn’t fit with the way these women thought or talked; the words were unnatural and detracted from the rest of the messaging and visuals. 

When the researchers discovered this insight, the creative team quickly borrowed more pleasing copy from another version and created a new iteration to show the community, which they immediately applauded. Within two days they were back in field conducting another round of quantitative evaluation and this time, the newly developed version clearly came out on top, giving the client a winning, fully optimized version to build into the marketing campaign. 

In another example of how hybrid made the difference between a winning campaign and a disaster, a client was testing a somewhat controversial set of online and TV ads; these ads attempted to push the boundary with their language and included cultural references the brand believed would greatly attract the attention of the desired target, a group of ethnic minorities. In order to walk the fine line between highly relevant and racially insensitive, the research team chose to test these ads in a way that allowed them to get a complete picture of the audience’s response; not only to identify which ad variation was most impactful but to also gauge whether there was any notion of “crossing the line” with the target group. Digital-moderated intercept chats at the end of the survey allowed the researchers to hear directly from the respondents who viewed the clips, confirming that while one ad version tested highest in the group, it was also the one most likely to be called out as inappropriate. This allowed the client to choose the next-best alternative, thereby avoiding the language that was just a bit over the line and a potential public relations nightmare. 

Inspiration for next phase

One of the most intriguing side effects experienced by researchers who are leveraging hybrid quant+qual research is that they often find themselves with an abundance of riches for the next iteration of their work. Whether they integrated quick consumer video responses, included hybrid chat intercepts or linked their quant to a pop-up community for deeper insights, it’s common for marketers to find their next germ of an idea or the starting point for an entirely new campaign from within the qualitative stories they gathered. A respondent might say something unique or share a clever phrasing of the tagline; these “cutting-room floor” learnings often become the inspiration for the next phase of creative work.

While there are many examples of how these additional learnings help build the foundation for the next phases of research, one unique case occurred during a hybrid customer experience project. The client was a major grocery store chain that was evaluating the customer experience of a new prototype service feature of the store and the messaging to support and promote it. While the feedback in the quantitative component was mostly positive, during the qualitative interviews conducted in real time at the end of the survey, many respondents remarked that they actually had trouble finding this particular new feature within the store; it had been placed towards the front, near customer service, but many of the shoppers interviewed noted that they would have expected it reside within a different section. Additionally, the service’s trade name was clever and unique but it and the messaging around it didn’t accurately reflect what the service was actually about. 

Neither of these issues were discovered in the quantitative phase and were not a planned component of the qualitative interviews either. However, this unexpected insight led researchers down a path of questioning whether placement of this service was going to be a hindrance to its eventual adoption and whether there were better ways to promote the service as they prepped for the nationwide rollout. Had the researchers skipped the hybrid qualitative piece, they would have missed the critical feedback that led to the service’s eventual renaming and repositioning. 

Stepping back into the spotlight 

Finally, after a long turn out of favor, we see hybrid approaches to ad, concept and message testing stepping back into the spotlight as a critical tool in the marketer’s toolbox. Statistically significant, quantitatively sound and accompanied by just the right amount of qualitative context, hybrid approaches enhance understanding, confidence and success rates with much needed data and context – all without adding complexity to the project or significantly increasing timelines. Past reasons to exclude combined research methods have been solved by technology and get better every day. What are you waiting for?