Don’t get ahead of yourself
Editor's note: Susan Fader is a qualitative researcher at FaderFocus. She can be reached at susanfader@faderfocus.com.
To begin designing or even thinking about starting any research journey, you need to make baseline assumptions. But with the current emphasis on agile timelines – which means getting into and out of the field as quickly as possible – little to no time is spent on evaluating if the assumptions used in prior rounds are the right fit for the research at hand. In many cases, no evaluation is done and past assumptions are just taken as givens.
The trouble is, if you start your research relying on the faulty baseline assumptions, you can end up going down the wrong research path and, even worse, unknowingly end up drawing inaccurate conclusions – no matter how much data you are able to collect during fielding.
For example, until about 500 years ago, astronomers believed that the planets and the sun revolved around Earth. All their observations, data-gathering and analysis of planetary rotation paths and movements of stars were based on a very incorrect baseline assumption. Under their Earth-centric view, the planetary movements seemed to follow erratic rotation patterns. But even so, since they felt that their baseline assumption was the right starting point for gathering their data, they continued to amass information and draw incorrect conclusions.
Then Nicolaus Copernicus recontextualized how astronomers should be analyzing planetary rotation. He did this by changing the baseline assumption and revolutionized the science of astronomy by postulating that all the planets – including the Earth – revolved around the sun. This belief provided observational data that mapped out a radically different view of planetary rotation and also made much more sense.
The world of market research can also learn from the Copernicus example. Right now the focus, in market research, is almost exclusively on “agile” through incorporating technological innovations that can accelerate the executional/fielding and analytical stages. In many cases, researchers don’t even realize that they are speeding through the important contextual stage of research by quickly reverting to and accepting “givens” – namely, using previous research findings, existing/historical demographic segments and accompanying stereotypes as the foundation for baseline assumptions.
To reinforce the importance and need to focus more attention on the contextual stage, I like to think of market research as having a similar structure to running a marathon (Figure 1). Hardly anyone who is running a marathon would just show up to the starting line without spending prior time reviewing and reevaluating their past assumptions/possible running strategies. Just because a marathon strategy may have worked before doesn’t mean the same strategy will work for the next one, because many key elements that can impact how they should run could be different, e.g., the course route and elevation, the weather, the runner’s current fitness and who the competition is.
When you incorporate the contextual stage into your research project, you are using what I call contextual intelligence, which helps make sure the research is designed and executed using accurate assumptions, giving you the right starting point and context to frame the research challenge and begin your exploration.
Just like running a race, the contextual intelligence stage of research also has three main components:
What: the objectives to be laid out
Who: the demographics of who to talk to
Where: the areas the research should cover and how the conversation should begin/flow
What
Without taking contextual intelligence and the totality of the overall context into account, you risk behaving like the blind men in the famous parable about their first encounter with an elephant. They are each only allowed to touch a specific small part of the elephant and thus come to very different and wildly inaccurate descriptions of what the animal actually looks like. Contextual intelligence is the flashlight that illuminates the whole elephant.
Context is also the prism we use to make judgement calls and decisions and it is important to recognize that context is not static. But rarely do we see contextual change in real time. Change may appear to be happening slowly and therefore not always be obvious, which is even more reason why we should be on the lookout.
A good example of the value of contextualizing the research objective comes from the world of hotel guest satisfaction studies. A hotel was puzzled by numerous complaints of slow elevators, even though the speeds of its elevators were comparable to industry standard. Adding an elevator bank would be expensive and time-consuming and require taking multiple rooms out of commission. Before going down that route, hotel management correctly decided to explore why guests were assuming the elevators were slow. (Note that this was before cellphones were common.) For guests the issue was not slow elevators but rather “being bored because I had nothing to do while I’m waiting.” So the correct starting point was “how to address the issue of guests being bored while waiting” versus “how to fix the problem of slow elevators.”
Ever wonder why hotel elevator banks are generally surrounded by a wall of mirrors? Well, nearly everyone likes looking at themselves and, when doing so, they are rarely bored! Take the time at the conceptual stage and make sure that you have the correct research “problem” in focus before you go into the field.
Who
As researchers, we like to think we are curious and open-minded. However, if we don’t understand the complex context of emotions, perceptions and experiences that shape people’s thoughts, actions and decisions then it’s quite possible we may be making inaccurate baseline assumptions.
Companies’ categorizations generally default along historical standard guidelines – in place since the 1960s – focusing on traditional demographics (age, gender, income, education, relationship status, kids, sexual orientation, etc.) with overlays of product/service product and usage. Sometimes attitudinal questions are added but they tend to be more fluff than code-breaking, e.g., “I am comfortable talking with people I don’t know.”
We need to recognize that categorizations/segmentations/demographic groupings are generally designed to fit potential consumers into structures that meet business-unit needs; they’re not necessarily how the consumers see themselves.
When it comes to who we recruit for research we need to incorporate what I call the cognitive demographics component of contextual intelligence, which is about recognizing how people self-define versus putting them into demographic categories predefined by marketers or researchers.
The importance of contextualizing identity prior to designing a research study – more specifically, how you define identity – is imperative to elevate engagement and participation, especially in qualitative research. It is one of the many codes that make up that “neuro-combination lock” on our individual mind-safes. Without taking into account how respondents self-define, the research could be wrongly interpreted.
Let’s look at a study I did with moms about products they might purchase for their family. The traditional demographic approach would identify the mothers as being the same, while cognitive demographics would show they are different.
For example, two recruited moms are the same age, have the same number of children of the same genders and ages, have same household income, the same profession, go to the same church and in fact are neighbors. Does that mean they should be grouped as demographically similar? Actually no. The two moms perceive their roles as moms very differently. During recruiting, when I asked them what their “mom motto” was, I got very different answers. One mom’s motto was a laid-back “I go with the flow,” while the other’s was a more helicopter-mom-like “I will do anything for my kids.”
Without taking into account how they self-define and perceive their worlds, these moms’ individual feedback during the research could be wrongly interpreted. From a cognitive demographic perspective, they see themselves as having very different views on motherhood and therefore will make purchase decisions differently.
In addition, how companies have historically asked demographic screening and categorization questions may no longer be appropriate when it comes to aspects like race and gender that were previously rigidly defined. A 2022 study, How to Ask Race and Ethnicity in a More Inclusive and Sensitive Way, underwritten by 11 different, diverse research companies and overseen by the Insights Association, explored the appropriate ways to ask what is generally the demographic laundry list.
One of the important takeaways was to consider if you even need to pose all of the typical demographic questions. If they’re not relevant to the study, just drop them.
Race is a contentious topic in the U.S. and, as the study showed, asking people to put themselves into preselected categories can be off-putting, especially when over 10% of all Americans now consider themselves multiracial (2020 U.S. Census Bureau) and thus have difficulty choosing only one racial designation.
The Insights Association study also showed that trying to be all-inclusive by providing an abundance of racial and gender options can also have negative ramifications. Check it out for further demographic screening guideline insights.
Where
Too often the amount of material the research study needs to cover overflows the discussion or questionnaire time frame and there is strong temptation to say, “We already know the answers from past research so we don’t need to include any probing around this in this research study.” But just because you – the researcher/company – know something from past research doesn’t mean it is top of mind to consumers or still relevant or as important as you may think. Even your most frequent users may not have your product or service top of mind.
I have found that adding a pre-work/homework assignment that I call a self-diagnostic ethnography can be extremely beneficial in generating in-depth and thoughtful conversations during the actual research discussion. A self-diagnostic ethnography is the anthesis of a data collection exercise, which tends to be positioned as, “We don’t have time to ask in the discussion so let’s just have them answer the questions as a homework assignment.” Its purpose is to create a situation where the research participant can self-observe their behavior in their natural environment before they come to the research discussion. It allows them to consciously observe and analyze actions that are normally automatic and bring that newfound awareness to the research discussion.
If it is a qualitative research study, I like to think of the overall structure of the discussion as a long hallway with many doors. Each door is numbered and behind each door is a specific area of questions. A typical qualitative study involves the moderator metaphorically taking the person’s hand and exploring each door in a prescribed order. First the questions behind door number one, then the questions behind door number two, etc.
However, if you allow the person to initially lead the discussion they might go to door nine first, then door four, then to door three or even bring you to a door you didn’t know existed but is integral to their worldview. The journey they take, what they emphasize, what they leave out, etc., can not only be very informative but generate game-changing insights. More importantly this open-ended storytelling structure, where the participant – instead of the moderator/interviewer – determines how they share their story about the research topic you are exploring, will generate more relevant and deeper insights than a traditional question/answer-structured guide. A bonus is that you can often cover more material because a lot of information is provided in unaided storytelling form, rather than requiring the moderator to ask a laundry list of questions.
An example of this was for a study I did on fabric softener new product ideas, where participants, all heavy users of the brand in question, completed a self-diagnostic ethnography exercise while doing laundry the week before the research study discussion. They were asked to record the three things they liked best and three things they liked least about doing laundry. Then, to start the research discussion, I asked them each to share their story of laundry likes and dislikes. Note that I specifically did not ask them to focus on the topic of the research – the fabric softener – because fabric softener resides in the world of laundry and I wanted to see how fabric softener would or would not come up organically in their laundry stories.
Surprisingly to the client, who thought they already knew what the consumers would say and so therefore didn’t need to hear them talk about how they do laundry, a number of loyal and heavy users of the company’s fabric softener didn’t mention fabric softener at all in their stories. It turned out that, for these consumers, adding fabric softener was such an automatic behavior that it was not really top of mind when they thought about doing laundry. This was an extremely important learning that impacted how the consumers perceived the new fabric softener product ideas and how new fabric softener products should be positioned to them and one we would not have gotten if we hadn’t taken the step of starting the discussion on laundry versus fabric softeners.
So, if you are doing a research study and your assumption is that your subject/product is top of mind with respondents because they are heavy users, your concept/new product discussion will be starting from the wrong place. In addition, you will aggravate the research context problem if you start the ice-breaker/warm-up with specific topic questions such as, ”Tell me why you use fabric softener.” But if you design your research project so that each consumer’s personal worldview of laundry – whether it does or does not contain fabric softener – is their personal starting line, then your research design will probably lead you to different conclusions of the viability and true interest in the new product ideas than if you started with a discussion focused on fabric softener.
Add another stage
Much of the current focus in marketing research is on the executional and analytical stages of a project. Even many of the recent technological innovations that have made our work more efficient – from dashboards to AI-driven data analysis tools – have focused on the post-field stages. I think we need to add another stage, a conceptual one that incorporates a contextual intelligence approach as a starting point to make sure we are really evaluating our baseline assumptions – before we go rushing ahead. The quest for agility is certainly worthwhile but by taking time early in the process, we can make sure we’ll be happy once we reach the finish line.