Do we need qualitative research to truly understand people?
Editor’s note: Tim Wragg is CEO of market research firm Human8.
After spending an estimated half a billion dollars surveying voters across the United States, with cutting-edge technologies and methodologies, the truth is staring us in the face yet again. We got it wrong. Pollsters, market researchers, forecasters and insights gurus all predicted things that never happened and were unaware of things that did. The disconnect between research and reality – something I hear from many business leaders – is as stark as ever.
Well, up to a point.
Yes, industry experts underestimated Donald Trump’s popularity across the board, especially with women voters, young males and Latinos. Online panels and phone surveys missed huge swings toward Trump throughout the country, particularly in once-staunch Democrat districts. And how can it be that pollsters confidently predicted Kamala Harris had a four-point lead in Iowa, only for Trump to win the state by a whopping 13 points with almost 56% of the vote?
After misjudging the strength of Trump’s support in both the 2016 and 2020 elections, pollsters recalibrated their quantitative data models to, in theory, more realistically reflect his standing. Clearly, they didn’t recalibrate enough. No one spotted what became a Trump landslide. Was it wishful thinking? Poll convergence? Or did we just miss the signals because we rushed to judgement (feeding the media’s insatiable appetite for new polls with hastily conducted surveys) instead of taking the time to understand people, talk to them, listen to them and observe their environments? Otherwise known as the qualitative side.
Political polls highlight the growing need for qualitative research
I believe there needs to be a fundamental shift in how the entire industry operates. Technical adjustments only go so far. For too long, we have relied upon single-mode polls, usually online or on the phone, that allow for inaccuracies and biases to play havoc with results. We have too much faith in quantitative data while quietly admitting that social norms affect the accuracy.
For instance, if you’re working double shifts to make ends meet (a likely Trump voter), you’re not going to have as much time to fill in a survey at leisure as younger people working from home might (typically, Harris voters). And considering the combustible nature of the election, it’s likely many people simply lied or stayed quiet about favoring Trump.
We have allowed our focus on quantitative measurement to take precedence over the qualitative. By that I mean the context in which these behaviors and beliefs exist. People’s everyday lives, their conversations with friends, their wavering emotions, the strength of their feelings and how they are affected by daily responsibilities, the issues that matter to them, what they love and loathe in their lives and to what extent, the context of people’s points of view. Their individual cultures. Unless we reach further inside these cultures, we’re never going to understand people as well as we think we can, no matter how sophisticated the technological bells and whistles at our disposal. Qualitative work, at scale, is key.
Otherwise, we end up imprisoned in our cozy bubbles. And, yes, I’m as much a victim as anyone. When I visited clients in the U.S. this year, I arrived convinced by media reports that Trump was a spent force. And then I drove through the country where I saw red MAGA caps everywhere, his face beaming from every front lawn, the car radio filled with Republican voters lauding their hero. Just a few days before the election, working in Detroit, I sat in a room filled with mostly Democrat women voters – the core of Harris’s vote – and each one solemnly said: “He’s going to win.”
On the ground, there was a strong sense that Trump had it in the bag. On the TV news later, I watched journalists interview political wonks, commentators, other journalists and former politicians for their insight into voting behaviors and, as they predicted a slim Harris victory, wondered, “When was the last time these people spent quality time with voters outside of their bubbles?”
On election morning, I read polling expert Nate Silver, founder of FiveThirtyEight, boast that he had run 80,000 models and that the election was “literally closer than a coin flip” but that Harris would probably edge it.
It doesn’t matter how many models you run, you can’t know people by asking them to tick a box and relying on highly reductive quantitative tools. Instead, you can only really understand people – whether it’s about the brands they buy or the votes they cast – by getting as close as you can to them.