Case studies show value of mixing traditional and technology-based qualitative approaches

By Marla Commons | May 1, 2007

Reading time: 8 minutes

Abstract

In separate projects involving financial services and alternative medicines, Research International found that a mix of traditional and new qualitative methods was the most effective approach.

Research Topics:: MCASI (Mobile Computer Aided Self Interviewing) | Qualitative Research
Industry/Market Focus:: Financial/Investment/Banks | Health Care (Healthcare) | Health Care Products-Natural
Content Type: Research Methodology

Share Print

Merging the old and the new

Editor’s note: Marla Commons is senior vice president, Qualitative Space, Research International, Chicago. This article is adapted from a presentation made at the ESOMAR qualitative conference in October 2006 and is reproduced with permission from ESOMAR.

It has become increasingly critical for the market researchers to provide deeper analysis, more compelling insights and highly actionable recommendations generated from the consumer research we conduct. The general response to these changing and stringent demands among qualitative researchers has been mixed and ranges from developing new approaches, offering ethnographic methodology, engaging in more projective techniques, offering technology as a solution and in some cases, making the decision to stand firm by positioning themselves as “traditional qualitative” shops (i.e., sticking to doing focus groups and facility-based research).

At the same time, there has also been a trend over the last few years away from conducting traditional qualitative approaches, toward the desire for approaches that are able to offer deeper connections with consumers.

“You get a different level of understanding when you’re standing in a consumer’s bathroom with them showing you what’s in their medicine cabinet. You start to understand their world and how your product fits or doesn’t with their needs. This is where those great insights start from. You get those ‘aha’ moments and know you’re on to something,” says a senior manager at a pharmaceutical firm

It appears the debate has shifted from merely discussing the significant value of qualitative research (especially in relation to quantitative) to that of discussing how to use new approaches and how to find ways of getting richer insights.

Instrumental factor

Often we find that new qualitative approaches or innovative techniques are equated with using or leveraging technology itself. This is perhaps due to the fact that technology has increasingly become an instrumental factor in the way our world in general operates. It impacts our lives both professionally and personally and has forever changed the way we think about communicating and connecting with each other.

Examples abound on how technology has changed qualitative research approaches. One significant advance is the invention of online focus groups. Others include “lurking” in blogspots, videotaping consumer behavior as it naturally occurs, using Webnography, providing video reports in addition to traditional written reports.

The pioneering spirit of qualitative researchers around the world will undoubtedly continue on the path of developing new approaches in qualitative research by using technology. So at this point it is important for us as a community to stop and ask ourselves what purpose these innovative approaches ultimately serve in the world of qualitative, including:

Do they selfishly serve to make us appear smart and innovative?
Do they simply serve as a new approach for those who are tired of traditional focus groups and facility-based research?
Do they ultimately better equip us with deep consumer insight?
Can they better inform subsequent quantitative research?
Should they be viewed merely as another option, one of many approaches in a toolbox of qualitative offerings?

These questions are important ones for us to consider, as we have a responsibility to recommend the best methodology/approach even when it’s tempting to do something different. The question becomes not whether we can provide new approaches - we have certainly proven that we can. The general question at hand appears to be whether traditional approaches we have used for years will fall out of fashion entirely or whether they will continue to play a significant role in consumer understanding. In order to address this question, our firm undertook a study to determine the real value that two completely different qualitative approaches can have on a study.

Case studies

We chose the areas of finance and health care as case studies to demonstrate how the right qualitative methodology can inform different yet equally compelling results in some cases, and ultimately, how different methodologies can inform different subsequent quantitative research questions. As shown in Figure 1, in Step 1 of these case studies, two completely different qualitative research methods were utilized to highlight how the qualitative methodology implemented impacts the level and type of insight generated.

The first approach utilized a more traditional qualitative methodology: in-depth, face-to-face interviews (IDIs). The second approach utilized Research International’s Cellnography approach in which consumers are given a camera and video-enabled cell phone and are asked to capture their behavior and feelings with the provided phone (i.e., “Please use your cell phone to take a video and describe what types of alternative medicines you use.”). Both of these methods were utilized in the bill payment and the alternative medicine case studies.

Traditional qualitative method: IDIs

For the bill payment IDIs, respondents were asked questions such as what bills are currently being paid, what methods of payment are utilized and why, and, importantly, questions surrounding respondents’ attitudes toward automatic and online payment options versus more traditional forms of payment such as check writing. They were also asked general questions about attitudes toward paying bills and were asked to describe their typical bill payment routine. For the alternative medicine portion of the IDIs, respondents were questioned about their awareness and use of different alternative medicine options, attitudes toward alternative and traditional medicines and future likelihood of adopting alternative medications.

New method: Cellnography

In both the bill payment and alternative medicine areas of investigation, respondents were given a packet of homework tasks containing several different assignments they could complete using the provided camera and video-enabled cell phones. All assignments were designed to capture respondents’ real-time behavior and feelings with the specific topic in mind. As they completed assignments, respondents uploaded and e-mailed back their photos, sound clips and videos.

Examples of assignments for the bill payment case study included having respondents video themselves during bill payment, asking about attitudes toward different bill payment options, and interviewing a friend or family member about how they pay bills in order to further understand how they view their own attitudes. Examples of assignments for the alternative medicine case study included asking about what alternative medicine means/how it is defined, usage of and experience with alternative medicine options and using video cameras to show where alternative medicines are stored in the home.

Results for both the IDI methodology findings and the Cellnography methodology findings for both the alternative medicine and the bill payment case studies were then analyzed and reported independently, yielding four separate qualitative reports. The contrasts found in the comparison of these reports make up a good part of the foundation from which our analysis and recommendations are drawn.

Advantages and limitations of IDIs/traditional qualitative research

IDIs certainly are the most appropriate qualitative approach when the facilitator needs to obtain in-depth and detailed information in a specific period of time (as evidenced in our case studies), especially in relation to focus groups where the individual may speak only occasionally.

Second - and this is perhaps the most critical reason for doing IDIs or focus groups - is the fact that these types of approaches enable researchers to understand consumers’ attitudes, perceptions, needs and wants - the original basis from which these approaches were applied in the first place.

In the age when qualitative research is sometimes used to make critical business decisions on whether to launch a product or whether to rename a service, the purpose and function of qualitative research can sometimes get lost. The purpose mistakenly becomes about measuring response to products or ideas rather than using the approach to explore reasons and motivations for responses. Comparing, as we did in this study, the results of new and traditional qualitative approaches helps us to enumerate the advantages and disadvantages of each methodology (Figure 2).

The question-response format of traditional qualitative methods such as IDIs ensures that specific and detailed information is gathered, yet it can also limit the ability for respondents to be spontaneous and more natural in their responses - simply because they are either in an unnatural context such as a research facility or in the case of ethnographic approaches because of the mere presence of the facilitator in one’s home.

Because qualitative researchers make a living from being able to read people and make them feel comfortable, it is sometimes easy to forget that despite best efforts, an inherent limitation in any facilitator-led research is the impact of their presence on the respondent. This is true in both facility and more natural-based ethnographic approaches and is an inherent limitation in qualitative research in most cases. With knowledge of social norms, respondents often feel the need to justify themselves before others to show that they know when they fall outside the boundaries of what others (including a moderator) may constitute as normal or acceptable behavior.

Finally, in traditional qualitative research, respondents are asked to explain their thoughts or behaviors, and as a result of the situation, their responses are usually conscious and methodical. Our attitudes are not always logical, though, and our decisions are not always made consciously - it is in these situations that innovative techniques are particularly relevant and can be used to better inform subsequent quantitative research.

Benefits and limitations of a new approach vs. traditional qualitative

Unlike traditional qualitative approaches Cellnography does not enable the researcher to probe in order to clarify responses or enable any sense of real control over the type of information provided. However, this lack of contact and constraint allows us to tap into a level of consumer understanding while at the same time avoiding having the moderator’s presence influence respondent behavior (despite the moderator’s best efforts to minimize this impact). Engaging respondents in a blog community or using a methodology like Cellnography can avoid respondent posturing and give accurate insights into actual behavior.

An additional benefit of an approach like Cellnography is its ability to reveal the underpinnings of emotional or subconscious decision-making processes, offer insight through a breadth of knowledge about a topic, and reveal behavior outside the constraints of social consciousness and verbalization. It also enables researchers to understand real-time consumer behavior rather than relying upon recalled/remembered behavior (Figure 3).

It depends

So, how do we select the right approach, whether traditional or new? This is one of the most frequently asked questions by clients. Our advice on the matter is often rather simple and perhaps not the direct answer that many would like: it depends on the objectives of the research.

Newer is not necessarily better, just as the traditional methods are not necessarily always going to be the right answer. In fact, our experience with this research suggests that it is perhaps the combination of both the traditional and the new that may provide clients with the deepest insights possible. In our case studies we derived insight as much from the findings unique to one methodology as from nuances in similar responses across approaches (Figure 4).

Consider the idea of a combination of facility-based interviews in conjunction with an approach like Cellnography or combining an ethnographic-based method with a facility-based technique as a way to provide more robust answers to research questions. Not only are the qualitative insights more fruitful and compelling, but there is the critical benefit of how these insights can powerfully inform subsequent quantitative work.