Still a valuable tool

Editor’s note: Jon Last is vice president, corporate marketing and research at The Golf Digest Companies, New York. Judith Langer is director, Roper/Langer Qualitative Division at RoperASW, a New York research company.

In the media as well as the marketing and advertising community, focus groups are increasingly drawing criticism. While 20 years ago it was quantitative researchers criticizing focus groups for not being scientific, now it is often qualitative researchers who claim the traditional or regular focus group is passé and inferior to other qualitative techniques. (Even the word “traditional” implies that the regular focus group is a thing of the past.)

The typical focus group format we refer to here has these elements: a moderator leading the discussion at a research facility with one-way mirror, eight to 10 respondents, one-and-a-half to two hours long, clients observe from behind the mirror and/or through remote viewing. (There are a number of variations of focus groups, including non-traditional ones that last for four or five hours, have 25 or even 50 respondents, have clients directly involved with respondents, take place outside of dedicated facilities, etc.)

Among the criticisms of the standard format are that it is stale, cold/clinical, inhibiting, lacking depth, and that it encourages overly rational and dishonest responses. In a March 2003 American Demographics article “The New Science of Focus Groups,” for instance, a critic who uses “game nights” among teenagers said the traditional focus group is a process he views as “a customer terrarium, with people behind glass...Focus groups are the crack cocaine of market research. You get hooked on them, and you’re afraid to make a move without them.”

At the same time, the article noted that focus groups continue to dominate qualitative research, citing numbers from Larry Gold, publisher of Inside Research, that show companies spent $1.1 billion on qualitative research in 2001, most of it for focus groups. When we asked Gold this fall what percentage of all qualitative research he estimates is accounted for by traditional focus groups, he said probably 85-90 percent.

So if focus groups are so outdated, why are they still such a frequently used technique? Do clients, the end users of focus groups, agree with the criticisms expressed by researchers with competing techniques and by the trade press? Why do they continue using focus groups and, looking ahead, do they anticipate continuing to use the technique in the future?

To explore the role of focus groups today, we conducted a study among past-year client users. We targeted them as people who could speak to the benefits and limitations of traditional focus groups within a broad context of other research and without any personal self-interest. Over 100 market researchers on the client side (advertisers and advertising agencies) across a range of industries were invited to participate. The sample was an amalgamation of research decision-makers culled from multiple sources, including a list of Advertising Research Foundation (ARF) members, past clients of Roper (not necessarily the Roper/Langer Qualitative Division), advertisers of The Golf Digest Companies and other clients. Over 60 researchers participated in the study, which was conducted in late June through mid-July via a self-directed questionnaire pushed via e-mail and hosted by QualTalk. (Thanks to Ted Kendall of QualTalk for all his assistance in setting up the study site and tabulating closed-ended answers.)

Continued vitality of traditional focus groups

Most of the study was open-ended, as befits our subject. We also included some close-ended questions to provide a directional sense of recent and anticipated usage of various qualitative methodologies.

Using current 2003 and last year’s research budgets in tandem, respondents indicated that the mix of methodologies they use is weighted slightly to the quantitative side. Less than half indicated that there had been a recent shift towards using slightly more quantitative (defined as 5-10 percent more) or significantly more quantitative research over the past three years. In fact, over a third (10 percent+) of those interviewed indicated that they had been shifting their research mix more towards the qualitative side over the same time period.

Not only had respondents used traditional focus groups in the past year, but the technique was by far the methodology they used most over the course of their careers. Of other qualitative methods, only in-person depth interviews were utilized by more than half the respondents in the past year. Phone depth interviews, dyads and triads, and ethnography were each used by between a third and four out of 10 during the past year.

A more telling demonstration of the popularity of the traditional focus group is seen in respondents’ budget allocations for 2002-3 compared to expenditures on alternative qualitative methodologies. The traditional focus group commanded nearly two-thirds of these end users’ budgets on average. Only in-person depth interviews and non-traditional focus groups received greater than 6 percent of the qualitative budget, and they were the second- and third-most popular techniques, respectively.

Asked if the traditional focus group methodology is more relevant, less relevant or no different in relevance than it was five years ago, nearly nine of 10 clients interviewed said its relevance is unchanged (64 percent) or greater (25 percent) today.

The traditional focus group has support from research professionals as much as from general marketers, according to respondents. The idea that its popularity stems from management’s embracing its simplicity and accessibility may be true. However, respondents indicated that they, as market research directors, are most typically the force driving the decision to use focus groups. A sizeable seven out of 10 of the decisions to use focus groups came from internal research departments or agencies rather than from marketing or brand management, respondents said.

Alternative qualitative methodologies gain in popularity

While this study points to a continued reliance upon traditional focus groups, respondents display a familiarity and growing comfort level with several alternative techniques. We asked end users which of a list of qualitative methods they or a colleague had utilized and found to be more productive/useful than focus groups. Ethnographic or observational research has been used at some point by half of the respondents and in-depth interviews conducted by telephone were cited by more than a third of the respondents, while online qualitative research (defined as live chats, e-mail or bulletin boards) are seen as useful by nearly two out of 10 end users. However, more than a quarter of those interviewed said no other qualitative technique is more productive or useful than the traditional focus group.

A look ahead at qualitative research usage

More than three-quarters of participants indicated they expect to use non-traditional qualitative techniques more in the future. But, again, our findings point to a preeminent place for the traditional focus group in the years ahead. Slightly over two-thirds of those surveyed indicated that they plan to continue using the method as much over the next few years as they have in the recent past. In short, clients will expand the type of qualitative methods they use.

Focus groups = qualitative research

Answers to the in-depth open-ended questions in our survey help to explain both why the traditional focus group continues to hold a prominent place among market research techniques and why some alternatives are attractive.

For most of the clients interviewed, focus groups are qualitative research. These market researchers know that qualitative research is a lot more than just focus groups, but their internal clients do not always make the distinction and may not even be aware that other qualitative approaches exist. The request from the researchers’ clients for qualitative research is often expressed as “We want focus groups.” Obviously, this may be one reason why focus groups are used more than other qualitative methodologies.

The identification of focus groups with qualitative research also results in FGs being blamed for whatever limitations qualitative has by non-researcher clients - even when client researchers and qualitative practitioners counsel otherwise.

Increasingly, clients want and need hard numbers, respondents said. While focus groups continue to provide insights, several problems arise:

  • Marketers are sometimes frustrated not to have projectable findings. In effect, they criticize qualitative research for not being quantitative research.
  • Focus groups are sometimes used as a quantitative research substitute because they are (relatively) quick and less expensive than a full-scale survey. While this is hardly new, the increased pressure to deliver research faster and cheaper can contribute to the misuse of focus groups. The hypotheses coming out of a focus group study are taken by some marketers as being definitive answers.

“Traditional focus groups are being replaced by online surveys and telephone interviews for a number of applications in my company...The biggest cavil about traditional focus groups is that they aren’t representative. Our numbers-oriented management doesn’t trust them.” - Isobel Osius, Meredith Corporation

“Our greatest obstacle is, people want to use [focus groups] instead of quantitative research and then they shy away when they are reminded of the non-scientific way participants are selected and that they are not ‘representative.’ ” - respondent asked to remain anonymous

FGs as a whole are taken to task when inadequate, unprepared or inexperienced moderators do a poor job. The technique, some respondents said, is highly dependent on the quality of the moderator. More sophisticated market researchers recognize differences among moderators, but non-researchers may assume the problem is focus groups in general.

Fieldwork issues affect the credibility of FGs too, a few respondents said. Concerns about professional respondents and poorly recruited groups were expressed by some clients.

“The recruiting has to be dead-on. When we’ve had nightmares, it’s due to the loose recruiting by the research company. It was a complete waste.” - Sara Killeen, Nike Golf

“Bad groups I’ve observed...have been more about a moderator who wasn’t skilled and simply asked questions from a questionnaire-type guide and never probed ‘why’ to any response.” - Kathy Canady, The Buntin Group

Clients themselves are to blame for FGs failing, some respondents said. The study sometimes is not thought out enough on the client side, especially when the objectives are not agreed upon. Sometimes the FG process goes well but clients refuse to listen, hearing only what they want to believe. Most striking, some respondents described outright misuses of FGs by clients, such as clients trying to sell respondents on their products or hand-picking favorable respondents.

“Some members of management often argue that we won’t learn anything new. Those same people will also say upon our return, ‘I could have told you that.’ No respect for what the consumer has to say and how they articulate it.” - anonymous

Why do clients use traditional focus groups?

Focus groups are perhaps even more relevant than in the past, according to some respondents, because of the advantages that qualitative research in general provides. Qualitative insights and connection with consumers are needed more than ever, they asserted. Marketers are competing to stand out in an increasingly commoditized world and have also become more removed from real people. They want and need to know the reasons why consumers feel and behave as they do.

While other qualitative techniques may offer these advantages too, FGs are seen by some as an excellent approach for delivering insights and close-up views of consumers and business customers. Specific benefits of FGs respondents talked about include:

  • Client comfort. Many clients are both familiar and satisfied with the FG process. A number enjoy - and even love - watching FGs. This comfort in turn makes it easier for advertiser and agency researchers to get acceptance for the use of FGs.
  • Cost-effective. FGs are less expensive and more efficient than other qualitative methods, some said.
  • Firsthand experience. Viewing FGs from behind the one-way mirror gives clients a feel for consumers and for their own customers. Viewers “get it” on an emotional level. (Interestingly, this advantage was not mentioned for in-person depth interviews.)
  • The group behind the group. The experience of clients watching FGs together process has a value in itself. Normally, busy executives are brought together to focus, quite literally, on an issue, typically in an environment free of interruptions from the outside.

“Many researchers in our company prefer qualitative research because they find it more fun, like to travel, and enjoy the richness of information it produces.” - anonymous

“Most people love groups. You get that up-close-and-personal feeling. You’re no longer just talking about the faceless ‘consumer.’ ” - Julie Medalis, Sylvan Learning Center

“Time/logistics. Financial, most definitely. We can do groups much less expensively than any other technique.” - Kathy Canady, The Buntin Group

FGs are and should be used as a first step prior to quantitative research, for exploration rather than decision-making, respondents agreed. At this stage, clients may not even have a clear picture of what their problem is. Among the specific uses mentioned are, in respondents’ words:

  • Exploring a new category or in trying to understand the underlying drivers of consumer behavior.
  • Developing theories or hypotheses to be tested (quantitatively).
  • Early exploration of ideas to better understand hot buttons.
  • To develop concept phraseology before quantitative.
  • To physically show new products or advertising and get visceral reaction to executions, or suggestions for improvement. To help your clients visualize their product being used by actual consumers.
  • Disaster checks.
  • To back up depth information that was not revealed in a quantitative study.
  • Putting a face on the consumer and bringing them to life.
  • Use to probe emotional issues, to compare strategic ideas, to explore unknowns, to find ideas for strategies.

Other uses that go beyond research per se include use of FG video clips as a training tool for internal personnel and for public relations purposes; a loyalty tool in itself for a retailer to show the store’s concern for customers’ views.

Non-research problems with focus groups

Practical and/or political considerations play a role in clients’ decision not to use FGs, even among these past-year users.

  • Money. Online quantitative research is seen as cheaper and clients may not be appreciative of the quality differences, some respondents said. Since quantification is needed with qualitative research, some companies would rather skip to that phase, leaving out any qualitative research.
  • Time and money. These are interrelated reasons: attending in-person focus groups take a lot of both, resources that companies are short on today. Remote viewing (which also costs money) is not necessarily accepted as a good substitute.
  • Boredom. Some observers find the FG format too static. Perhaps after watching FGs for years, they want something more exciting.
  • Past problems. Internal clients who - rightly or wrongly - were disappointed with previous FG studies may veto the use of the method.

“Time is the enemy. No one wishes to commit to the travel and sometimes this restricts the willingness to go to more distant markets. My clients aren’t as interested in video-technology, still feel being there firsthand somehow enhances their experience.” - anonymous

“Focus groups can be expensive and time-consuming to plan and execute. We are resource-constrained on both — therefore, there has to be a good reason to conduct them.” - anonymous

“Start with budget, which always shapes the decision on what methodologies are used. Then add individual preferences, such as a refusal to do focus groups because of a past negative experience. Throw in time pressures and naive research users who have heard that the Internet is faster and cheaper than anything else. Then add the fact that you generally have to do quantitative anyway, to back up the business decision, and it can be difficult to talk people into seeing the value of focus groups as a first step in the research process...” - Isobel Osius, Meredith Corporation

“Sometimes it is a battle to get marketers/salespeople to agree to focus groups, because they ‘seem so static and boring.’ What I’ve realized, the problem is this: [some] marketers and salespeople have a very highly developed sense of entertainment, and they’re simply bored by the process. Even though focus groups work (if they’re done right) and valuable info is gained by them, some of them have trouble sitting still long enough to realize that they’re working. This is probably the longest these people have sat in one room in the past 10 years! The MTV effect is definitely happening here as well, as more and more executives expect flashing lights, a grooving soundtrack and special effects at all times. So, they get pulled in by all the exciting bells and whistles that some companies offer and ignore the depth of info they COULD be getting with traditional focus groups.” - Susan Walsh, Hearst Magazines

“One tool in the toolbox”

This description of traditional focus groups as just one of many techniques was repeated by several respondents. FGs are and should be used when they are appropriate to the objectives of the particular research study. They are not meant to be an all-purpose approach; the fact that other qualitative techniques may be more appropriate in certain cases does not mean the focus group is a bad method.

Variations of group interviews - dyads, triads, mini-groups - and individual depth interviews in-person or by phone are qualitative alternatives that some respondents prefer at times. These techniques have been around a long time too, putting them in the “traditional” category along with focus groups. For some clients, the issue is not either/or - they use a combination of these methods on an individual project or over the course of the year. “I don’t see it as focus groups vs. IDIs but just ‘qualitative,’ ” one respondent wrote.

Advantages of IDIs and smaller group interviews cited by some respondents include:

  • Greater depth because each respondent is interviewed at greater length than in a FG; more participation by each respondent.
  • More appropriate for sensitive topics.
  • Avoiding group bias, dominant respondent problem of FGs. Interestingly, while this issue has often been mentioned in articles over the years, only a few study participants brought it up as a FG negative.
  • Better for communications research on advertising and concepts, since reactions in real life are typically on an individual level.
  • Better for executive-level respondents who don’t have time or desire to be part of a larger group or do not want to travel to a meeting location.
  • Sometimes the only practical choice, e.g., when respondents are geographically spread out and FG recruiting would be impossible.
  • Less expensive than FGs because of fewer respondents and, in the case of telephone depth interviews, no travel costs.

“In-depth interviews in person give a more complete picture due in part to the interviewer’s ability to observe discrepancies between what the respondent says and what can be observed on site.” - anonymous

“Once we’ve decided on qualitative, the decision comes down to small (one-on-one, triad) vs. larger (mini-groups or groups). This choice is usually driven by the sensitivity of the topic, the need to provide complete privacy, the desire for group interaction, and the potential for group bias. Recently used a mix of groups and IDIs to explore issues of race, class and education. The combination was critical in seeing that people were extremely uncomfortable speaking in what they feared were racist of classist terms. But the same concerns emerged in both settings, helping us feel they were real and powerful.” - Julie Medalis, Sylvan Learning Center

“We don’t use traditional focus groups as much as we used to. We have reduced the size of our traditional type groups to four or five respondents. Smaller groups are easier to manage and, more frequently, all respondents participate.” - anonymous

“One-on-ones are more times than not the better choice for B2B research: wanting to get the ‘facts’ and not the face nor body language nor interactive response. Also these respondents are not likely located in close proximity to each other.” - Tom Flint, FedEx Services

“Often a group environment can move individuals towards the thoughts of the ‘leader’ in the group. They find common ground too quickly, and it may not reflect the range of opinions on a topic.” - anonymous

Ethnography sparks interest

Among the newer or newly popular alternative approaches, ethnography sparks the most interest as an alternative and, in some cases, a complement to FGs. Several clients said they have used ethnography successfully and several others talked about wanting to use it.

Observing people in their own environments offers a more “honest” view of what people really do, rather than relying on their verbal reports, some respondents stated. Interviews in respondents’ homes and other natural settings also put people at ease and encourage greater candor than the “forced” facility environment, some respondents said.

“With the observational research we were able to observe without the ‘respondents’ knowing we were observing or what we wanted them to do. Sometimes people in focus groups try and give what they think is the right answer we are looking for.” - anonymous

“The camera doesn’t lie — observation sometimes speaks volumes as to how someone truly feels or uses a particular product. I wish I could do more of this.” - Kurt Lowell, L’Oreal

“Identification of new consumers’ needs — ethnography research is better. Understanding of potential product issues after launch (in-home visits could be a good idea)...Interaction with the product and the real environment situation.” - Soledad Miguel, H.J. Heinz

A variety of techniques within traditional focus groups can enhance their value, integrating some of the benefits of observational research, a few clients noted.

“[I] have set up mini-groups in which consumers were asked to create collages at home related to the given topic, or take photos of the environment in which the product was used. Can then do content analysis of what they bring to the interview. It’s a pretty cost-effective way of getting some more emotional/motivational and ethnographic insight.” -- anonymous

Online qualitative research, while not used by as many of the respondents, is seen as offering several key advantages. Like phone one-on-ones, this is a less expensive way of interviewing people all around the country, including hard-to-reach respondents. Logistically, online qualitative can be easier to execute than FGs, too. A few respondents specified a preference for bulletin boards over live chats.

“If a lot of regionality is needed, then perhaps online focus groups or bulletin boards are a better choice. Also, if the target is so obscure that it would be ineffective to try to recruit enough in one region to have groups.” -- Susan Walsh, Hearst Magazines

“Online bulletin boards are quick and the input can be much richer and more in-depth, if done right.” -- Isobel Osius, Meredith Corporation

“Moving ahead, I do think the nature of our qualitative work will evolve, with more participation by Web, more online, etc. This is more easy to swallow in terms of time commitment and money. The trade-off is that you lose some immediacy of feeling, and the ability to sit as a client group to think and react together.” -- Julie Medalis, Sylvan Learning Center

The future
Our research suggests that rather than becoming research dinosaurs, traditional focus groups may remain a — or the — major qualitative method in the near future. Clients find value in the process, are comfortable with it and think it is efficient in many cases. They have strongly defined and articulated opinions regarding the benefits and limitations of the methodology. They recognize that focus groups, when conducted with a skilled moderator and among a well-recruited sample, can deliver breakthrough insights and go beyond the limitations of quantitative research by providing a deeper context from which marketers can assess concepts and ideas.

At the same time, the focus group’s share of the qualitative market seems likely to decrease in the near future. Such a shift may result from greater awareness among market researchers and their clients that traditional focus groups are one tool in a diverse toolbox of techniques, to be used instead of or in addition to other techniques — traditional and new — when it serves the research objectives.

Furthermore, for us, these findings suggest that despite the significant inroads that Internet data collection has made within the realm of marketing research as a whole, the online world will not replace the in-person focus group in the near future. Clients we interviewed continue to value the uniquely personal interaction of the focus group in providing flexibility and attaching a human face that the online world is still hard pressed to replicate.

This study also reinforces some other important and inherent advantages of FGs over quantitative approaches and even some other more expensive or extensive qualitative methods. That is their outward simplicity and sensory accessibility: the ease of understanding that can be derived when a client or management observes those magic moments of focus groups, where a participant articulates an emotion or idea that resonates with the rest of the group and, for that matter, with those behind the glass, who capture that moment as a justification of or springboard towards positioning a brand.

The focus group is there in front of all to see and experience. It is not buried within reams of crosstabulations or regression models. It is not lurking somewhere among a full day’s worth of IDIs or intercept interviews, but is packaged neatly within a 90-minute to two-hour time frame, exposed and often preserved in its rawest and purest form for many layers of management to see and grasp easily.

Some concern
There is concern, however, among some clients about the continued trend of diminishing response rates in market research, which affects focus groups as well as other methods. The professional respondent is a special worry for focus groups which can be amplified to the point of nullifying the findings in a traditional focus group setting.

And what of that traditional setting? While respondents did not overwhelmingly speak to concerns about the conference room setting of a traditional focus group facility and its potential to hinder open exchanges of ideas from group participants, some weak signals were apparent. Some of the shift towards ethnography and to IDIs is recognition of the limitations of focus groups and a quest towards a better way to obtain more honest qualitative insights.

Lastly, and perhaps most significantly, another pervasive theme is the belief that clients often minimize the impact of focus groups by expecting too much of them. This is exacerbated by decreasing budgets, which not only limit the number of groups that can be conducted — thus placing more and more weight on the groups that are held to produce useful results — but also don’t allow for quantitative validation via follow-up research.

Enduring place
The focus group seems to have reserved an enduring place in both marketing research and popular culture as our society continues to evolve (or, some may say, devolve) to an increasingly sound-byte-driven world. The metaphor of focus group as television program continues to have relevance. Even for some clients who have begun to embrace ethnography or other observational research techniques, the focus group still remains as a succinct voyeuristic window into the mindset of the elusive and often otherwise faceless customer.