Trendy method or essential tool?
Editor’s note: Judy Langer is president of New York-based Langer Qualitative. Jon Last is president of Sports & Leisure Research Group, White Plains, N.Y.
The qualitative research field is undergoing change at a more rapid pace than it has in its history, with new (or seemingly new) methodologies playing a greater role. Ethnography or observational research has drawn significant attention recently, not only in the market research and marketing fields but in the mass media as well. Referring to the consumers of his company’s products, Procter & Gamble CEO A.G. Lafley has championed “close observation of the boss, and her active participation in the process of innovation.” In his book The Game-Changer he states that this kind of study “all starts by doing something simple - keenly watching consumers, face-to-face, knee-to-knee and listening, with ears, eyes, heart, brain and your intuitive sixth sense.” That is, watching and listening in “immersions,” not from behind focus group facility one-way mirrors. Malcolm Gladwell famously advocated ethnography over focus groups in Blink, his best-seller, and in speeches to advertising audiences.
Ironically, the market research field does relatively little research on research. Rather than rely on hypothetical statements concerning the value of ethnography, we decided to ask research buyers for their opinions. The rationale was simple: clients have no vested interest in promoting specific research techniques, they just want methods that do the job. This study is a follow-up on one we conducted among clients in 2003 in which we found that the impending “death of focus groups” was premature and, at the same time, that interest in ethnography was definitely growing.
For our study we interviewed 26 research buyers and consultants by phone and/or e-mail; nearly all have been involved as clients in ethnography studies. They represent a wide range of fields.
Our findings suggest several key points concerning the client perspective on ethnography:
• Definitions of ethnography and observational research vary widely among respondents, in sometimes contradictory ways. Nevertheless the people we interviewed agree that in-context research has value to their organizations.
• Despite the perceived advantages, some serious obstacles exist to greater use of ethnography, even among its biggest fans.
• Clients express divided views on the desirability of using a professional ethnographer.
• So-called traditional qualitative research is used less now by some clients as they shift to ethnography. However, these methods have not been replaced and won’t be in the near-term future.
Ethnography’s increased popularity was attributed by several clients to the success stories in the media and at conferences, P&G’s leadership and to its general trendiness. A packaged-goods client said the company is “using ethnography a bit more than three years ago because our marketing teams are responding to the hype about the term in the marketing press.” A consultant noted that ethnography has “had a well-deserved growth curve. I heard it had successes in product introduction. It’s sort of the technique du jour. I think it plateaued. You see cycles in research that become big things and everyone wants to jump on the bandwagon to show how sophisticated they are; focus groups started that way.”
More pragmatic
Just what is ethnography? Classic ethnography of the academic kind, several clients noted, is grounded in social anthropology, entails participant/observer immersion for a long period and a cultural analysis framework. The market research version is more pragmatic, conducted over a much shorter time, often not by a trained anthropologist. “It’s not what social scientists would call ethnography,” a consultant-user stated. “You spend a few hours in a person’s house, not days. It’s a snapshot.”
Ethnography and observational research are not synonymous in a number of clients’ minds. An insurance industry client, for example, thinks of ethnography very broadly as employing visual and written approaches: “[It] would include most anything that involves collecting data in the [customer’s] natural environment. Observation, video ethnography, a day in the life, even diary studies - paper or video - could fall into the ethnographic bucket.”
An alcoholic-beverage client said, “Sometimes we do some informal interviews on the street with bartenders or specific consumer groups to bring some flavor and some dimension as a part of a presentation, to bring that kind life to it.”
Observation of people’s natural or demonstrated behavior - how they prepare coffee, surf the Internet, balance their bank accounts, store products in their medicine chest, process their company’s mail, what they talk about at a family dinner, and so on - is a key part of what a number of clients do in ethnographic studies.
A number of so-called ethnography studies described by clients do not include behavior observation. They are instead contextual or on-premise interviews in people’s habitats - at home, in the office, at a bar, shopping, at the golf course, etc. These are the real-life places where product usage and decision-making take place. To some extent, ethnography is defined by clients by what it isn’t: interviewing in traditional one-way mirror research facilities.
Where things get sticky is on regular Q&A interviews outside facilities that don’t make use of the setting for research purposes, except perhaps as background in the video. Girlfriend groups and depth interviews in so-called natural settings qualify in some clients’ minds. Other clients, however, vehemently argue that this isn’t ethnography; it’s just what used to be called an in-home/office/store interview.
One advertising/marketing services client felt misled by a research company. “This firm sold us on ethnography and we thought we’d watch how women do their lives day to day and their retail space, and we had women on their couch or in a coffee bar and it wasn’t what we signed up for. It was bad research.” A financial-services client stated, “I often hear on-site interviews in someone’s office described as ethnography. I disagree. Only if I do something directly related to that environment that I couldn’t do in a facility would I describe as ethnographic in nature.”
Positive experience
Leaving aside the contradictory definitions of ethnography, all the client-users interviewed reported having at least one positive experience with the methodology. They cite several ways in which ethnography has been of value.
Going deeper. The methodology is considered part of a broader trend in qualitative research to get below the surface and move closer to the consumer to attain deeper understanding of people’s real motivations, attitudes and behavior. Ethnography uncovers unarticulated needs, reveals actual behavior, motivations and emotions - things that researchers don’t know to ask, that people don’t know about themselves and that they don’t want to admit to others. It goes beyond the rational reasons people make up to justify themselves. “We have [consumers] open their pantry and tell us why they chose to stock things where they did,” a consultant explained. “They may say they’re health-and-wellness-oriented but we see things that are not articulated in another interview.”
Viewer impact. The level of understanding can be greater from watching ethnographic interviews, whether firsthand or on tape, than from other research. Vivid video brings consumers to life in dramatic demonstrations of actual consumer behavior and is a great way to communicate ideas to internal and external clients and to ad agencies. The same points might come out in other qualitative research, some clients said, but ethnography helps them “get it” on a gut level.
A consultant previously at an office-products company explained, “You get a depth of feeling for what the customer does that goes well beyond the verbal or written description. Watch the administrative assistants going out of their minds. [We knew] this, but until you see how they do it, how they accommodate to the task you don’t realize how bad it is. You get more of a personal appreciation of the pain they’re going through.” Along those same lines, a media client said, “Showing our internal stakeholders the raw video had much more impact than any data points I could provide.”
For corporate managers, in-context interviews with customers can hit home. A beverage client stated, “It’s more like an in-depth interview than a focus group to get as close to the individual setting as possible, as opposed to bringing them out of their environment. Senior managers have no idea of what the regular consumer is; they’re living on their half-million dollar salary and have no idea of the people living on a $45,000 income.”
More real. Responses, several clients believe, are better when people are interviewed in their own environment than in a facility. They are more honest and reveal more because their inhibitions are lowered. The environment jogs their memories and enables them to demonstrate their real behavior. “The key advantage is the ability to ‘show me’ vs. just ‘tell me.’ While you can have respondents bring some things to a facility, they certainly can’t replicate their entire home or work environments. I’ve actually gone into fields with farmers [and had] respondents really show how something works or something they have difficulty with right in the place where they would naturally use it,” a financial services client explained.
Another client said, “Consumers can be inhibited when discussing alcoholic beverages in a group setting so the ethnographic approach makes the conversation more natural and allows for richer insights. By conducting discussions in the consumer’s home, either one-on-one or with a group of friends recruited by the consumer, the ethnography has more relaxed, natural setting that’s more conducive to uninhibited feedback.”
Major obstacles
So, why don’t clients use ethnography more? While some clients are frequent ethnography users, others employ it relatively infrequently, despite finding it valuable. A combination of major obstacles get in the way.
Time. Ethnography is labor-intensive for clients involved in the observation and, even more, in the analysis. The market researcher and internal clients spend several full days out of their offices interviewing/observing and traveling. In contrast, facility-based focus groups or depth interviews make it possible to interview more people per day and, of course, respondents come to where the client is. “Dud” respondents and no-shows in ethnography studies waste time because there is no backup respondent in waiting. Not only are facility interviews more efficient, they can also be better, a media client said. “Sometimes you take someone out of the environment and you can focus more. IDIs can be done in sequence, from 9 a.m. to 10 p.m. for two days.”
Reviewing, editing and analyzing videos is highly time-consuming - and difficult. “Ethnography is wonderful but takes forever,” a financial services client said. “The real time thing is making sense of them when you come back. You have to see patterns; in focus groups it’s usually apparent but with ethnography, pattern recognition often takes more time.”
Money. Ethnography’s cost-per-interview is greater than other qualitative methods like focus groups, and the total study price is often high, even with fewer interviews. Travel expenses and “fancy filming” push costs up. A media client explained, “It’s primarily a budget issue. There’s just a lot less interest in big-budget, exploratory research. Ethnography doesn’t stand on its own as a single project, but can bring a lot of value to a larger project. Unfortunately, it’s viewed as a nice-to-have rather than the nuts and bolts of necessity.” Another media client put it succinctly: “Mostly, it’s a cost/value decision. Big investment to do ethnos, so the need really has to be there.”
Convincing management. Some clients would love to use ethnography but cite upper management’s resistance to methodology that is unfamiliar and has an esoteric image. “[We] have been trying to educate our internal clients about the value of ethnographic research,” a media client explained. “We’ve come close on a few projects but nothing has materialized yet. I think this is due to the costs, the long timelines, the desire for numbers that can be easily dropped into PowerPoint slides and thus taken to market quickly vs. videotapes from ethnographic research requiring additional time and money to produce a market-ready tool, [and] internal clients’ uncertainty about how they would use ethnographic results with their clients.”
Samples. In the current research climate, so sensitive regarding data quality, some clients raised concerns about respondent authenticity, amplifying the oxymoronic client concern that qualitative research be more “representative.” Even judged against other qualitative methods, the sample size for ethnography is usually very small. “The sentiment is, do we feel comfortable just doing six interviews?” a beverage client explained. “There’s comfort with numbers even though the information may be much richer [with ethnography].”
A media client raised other questions about participants: “The sample may not be representative. Who allows a stranger to come into your house? ‘Can I go shopping with you? Watch you doing your laundry? Where you keep your toilet paper?’ Are these people more trusting and outgoing? More desperate to be listened to? More likely to think their opinions count?” Recruiting for ethnographies can be “super-intensive,” some clients noted. People are often hesitant to have strangers come to and spend hours in their home or office, watching and recording their activities, disrupting their routines.
Pigeonholing. Ethnography’s value in broad exploration seems to limit some clients’ use of the methodology. “A few years ago we were working hard to understand our consumer targets as people. Ethnography was great for that,” a beverage client said. “Now we are building on that understanding with other methods. I’m sure we’ll come back to ethnography in the future.”
Disappointment. While all the client users interviewed were happy with at least one ethnography project, some were let down in other cases. A pharmaceutical client felt “the ethnographer didn’t tie things together. [It was] a litany of what she saw vs. organizing the information in a holistic way that was meaningful.” In the case of a methodology still relatively new to clients, dissatisfaction with a project can affect future use more than with established methodologies.
True behavior
Some clients question whether ethnography does in fact reveal people’s true natural behavior and attitudes, which is supposed to be its key value. Participants may put on a show, try to make themselves sound good to interviewers (as they do in other types of research), clean their house before the visit, and so on. A golf equipment client said, “I can’t help but feel that there’s some real grandstanding going on. We’ll accompany a golfer into a retail environment, and he’ll pick up and rave about the benefits of the hot and costly brands. Yet, when we conduct purchase diaries, there’s a lot of knockoff product being purchased.”
Questioning the effect of observers, a media client who conducted informal girlfriend-type interviews commented, “It felt so contrived. Too many people were watching. The whole idea was it is more relaxed and I agree with that. But it’s clear that there are four people sitting along the wall doing nothing. If you’re hiding behind the mirror in a focus group, you disappear. But not if you’re taking notes [in the room].”
The reality issue may be caused by the client company who searches for the “ideal customer” and/or produces a highly-edited video that makes their brand look good. (Needless to say, this occurs in other types of research as well.) Some clients talked about going undercover to do their “ethnography,” pretending to be a fellow customer, grad student conducting research, etc. Clearly, there are ethical as well as methodological issues here.
Who should conduct ethnography?
Clients interviewed are sharply divided in their preferences concerning who should conduct ethnography: a professional ethnographer, a “regular” qualitative researcher or the clients themselves. Since market research’s version of ethnography is admittedly very different from the academic type, it may not be surprising that some clients do not feel a need for a professional ethnographer’s approach and cultural analysis. Some clients said what matters most is the individual researcher’s skills more than that person’s background. “The cultural anthropologist will probably do a better job than someone who doesn’t have the necessary skills,” one client said. “There are trained people who don’t do a good job and untrained people who have an instinct for asking questions in a non-intrusive manner.”
Cultural anthropologists and, to a lesser extent, researchers with an academic background in social sciences and psychology, bring special training to ethnography. A financial-services client believes that “an anthropologist or sociologist [uses] a toolkit grounded in social scientific models and theories, not gut instinct or impressionism. They use that toolkit to discern significant patterns - traces of culture - in the data. What I look for is a good ethnographer (not a degree!) who will promise a deliverable that gives me a concise model of client behavior that I can use to start solving a business problem.”
Other clients, however, do not think professional ethnographers are the best choice, as they may lack understanding of marketing in general and of the client’s product category specifically. Their approach is sometimes to “go broad, not deep,” pursuing topics the client considers irrelevant.
Qualitative researchers are preferred by several clients, especially if they have ethnography experience. Moderators are knowledgeable about marketing, probe relevant issues and have an established relationship with the client. “No specific training or background [is] sought, just excellent moderators who are great listeners and interpreters of consumer feedback,” one client explained.
On the negative side, a pharmaceutical client found that some qualitative researchers have trouble adjusting to ethnography. “I’ve run into in global situations with poorly-trained moderators. There’s not a really robust understanding of how observing can help a marketing person. Some moderators think they have to be very active and some of it is ‘show me your medicine cabinet.’ They don’t get it.”
Clients conduct ethnographies themselves in some cases. In addition to their market research experience, they have the advantage of knowing their category and brand. Some respondents, however, cited their lack of expertise in ethnography, scarce staff resources and time, and the fact that they may be too close to their product.
One method
Despite some pronouncements in the media that ethnography should replace traditional qualitative methods (focus groups and depth interviews, especially the former) none of the clients had dispensed with these approaches. Instead, they consider ethnography as one method in the qualitative toolkit. The increase in ethnography has indeed reduced several clients’ use of facility-based interviewing but has not replaced it:
“I don’t think ethnography will (or should) replace other types of qualitative research! I look upon ethnography as just another research method - it’s a means to an end.” (media client)
“Ethnography doesn’t replace focus groups. We utilize both and have gotten good at using each method.” (financial services client)
“Most people still have a comfort level with other [qualitative] approaches. They wouldn’t want them to be reduced. Also, the separate issues that are addressed by traditional methods haven’t gone away.” (beverage client)
Regular part
Focus groups are no longer the automatic qualitative choice for clients, especially early in the research process. Ethnography is now more a regular part of the research repertoire (along with in-person and phone depth interviews and online qualitative methods). “Many of my internal clients, instead of saying, ‘Go set up focus groups,’ said, ‘Go set up ethnographies’ because they got more insights. They got a taste for it in a real setting rather than in focus groups,” a consultant said. “The ethnography took the place of or reduced the use of focus groups to generate ideas, but once they needed to be developed or were in early rounds of testing, we’d use focus groups.”
Clients see the different methods as having different roles, sometimes on the same project:
• Ethnography for strategic understanding, exploratory research at the beginning of the creative/innovation process.
• Focus groups/IDIs for tactical issues - reactions to stimuli (ads, visuals, concepts, etc.), often ones that have been developed in the first phase. In a sense, these methods have been repositioned in some clients’ minds.
“I think of focus groups to get narrow things. If I have eight concepts, [I want to know] what’s working better than another, what language is working better, etc. When I’m trying to get something deeper and something I didn’t know how to ask, I would use one-on-ones. If I have the choice and the time - there’s time and money associated with ethnography - I’d opt for ethnography hands-down every time.” (pharmaceutical client)
“Focus groups still have their place for consensus-building and culling down ideas. For innovation, I do think ethnography has an advantage.” (consultant)
“[Ethnography’s] one of many methods, doesn’t come close to replacing traditional qualitative. Relatively little [research we do] is the exploratory stuff that ethnography’s good at. We do message development, ad testing and I don’t see ethnography doing that.” (pharmaceutical client)
“It’s rare that we do a study that doesn’t have both ethnography and focus groups. They both have their purposes. We try [ideas] out in the focus groups based on earlier insights and ideas drawn from the ethnography. Focus groups yield time and efficiency that can lead to tighter and more effective surveys.” (financial services client)
Continue to grow
Ethnography’s use will continue or grow over the next few years, some clients believe. “This big push is for deep consumer understanding, to look at the consumer landscape, to identify what consumer you go after and what the advertising should be. A real call to the researcher to bring these people to life,” a consultant said. (We also suspect that as ethnography becomes more of a mainstream methodology, senior management may become more receptive.)
Others think a plateau or drop-off may occur. Some large corporations already using ethnography extensively may not do more, and smaller companies are less likely to start because of budget issues. Disappointment with past projects puts some off from using/recommending it.
A key obstacle is today’s tough economy, a major issue for a method already seen as high-priced. “If everything were okay economy-wise, it could increase. Trying to know our people in a deeper way, that’s a trend, but ethnography is expensive. Things are hurting so it would be doing well just to stay the same,” a pharmaceutical client said. Interestingly, a beverage client said the tough times could work in the method’s favor: “Everyone’s experiencing the financial squeeze and research programs are being cut back, everyone’s operating with minimal budgets. In some cases that might mean the mega-projects will fall by the wayside and people will rely more on ethnography.”