Time-honored or time to go?

Editor’s note: Judith Langer is president of Langer Qualitative LLC, New York.

Here are excerpts from an article entitled “Fine-Tune Your Research” which appeared recently in Media: “Look for innovative ways to uncover insights and consider methodologies that are less well-worn and familiar than focus groups.” Also: “Focus group monomania: In many companies focus groups have become synonymous with qualitative research. While they’re suited to meet many objectives, focus groups also have distinct disadvantages - professional ‘focus groupers’ who have dominant personalities or submissive ones. Either can lead to misleading results.”

After reading the article, my reaction as a qualitative researcher (moderator) was a groan and a yawn. The yawn because this is one of many attacks routinely appearing these days in advertising, marketing and research trade publications (including this one) and in the general media as well. A groan because I believe that focus groups, with whatever limitations and shortcomings they have, remain a valuable tool under unfair attack. Long a staple of market research, focus groups are increasingly assailed. It’s a good time to ask if the method is passé or if it still has value.

Despite the frequent attacks, focus groups remain the most commonly used qualitative research technique both in the U.S.  and worldwide. Is there a solid reason for this or is it just because of client ignorance and inertia?

Pendulum has swung

Part of the continued popularity is the fact that, for good or bad, focus groups have become identified with qualitative research. Many clients regularly say “We want focus groups” when they really mean “Let’s do something to understand how consumers [or executives] really think and feel, what their underlying emotions are” (a more long-winded request, to be sure). To some extent, the pendulum has swung to what I call the “anything but focus group” side: “Give me ethnography/depth interviews/whatever, especially if it’s new and different.”

Focus groups are frequently described as “traditional” qualitative research - even though individual depth interviews (IDIs) and ethnography have also been done for decades. The term traditional is a loaded one, with negative connotations of being old-fashioned, out-of-date, rigid, etc. Worst, according to critics, is the traditional focus group format: sessions held in a one-way mirror facility with clients observing (eating M&Ms), six to 10 respondents around a conference table for two hours.

Inside and outside

Criticisms come both from inside and outside the research industry.

  • Qual vs. qual: Looking back to the 1970s when I started doing qualitative research, it was the quantitative researchers saying qualitative isn’t true research; articles on the “uses and abuses” were common. Now, it’s often qual researchers touting the superiority of alternative methods over traditional focus groups. Some of their methods are worthwhile, but others are just repackaged with new names and/or gimmicky.
  • Clients: Some now automatically reject the idea of focus groups (or, at least, anything called a focus group).
  • The media: Research and marketing publications run their own pieces and outside writers’ articles attacking focus groups. In the general media, it has become common to equate focus groups with all survey and market research and, worse, to imply that the method is only used to manipulate people and cater to the lowest level of tastes.

Perhaps the most prominent assault came from Malcolm Gladwell. His bestselling book Blink devotes a chapter to market research that specifically goes after focus groups. “Focus Groups Should be Abolished” was the attention-getting headline in the Advertising Age story in August 2005 that excerpted his address to the American Association of Advertising Agencies’ Account Planning conference.

Much of his criticism is based on the fact that the Aeron chair’s “aesthetic scores suck[ed]” in focus groups yet the Herman Miller company decided to introduce the style, which Gladwell applauds. The chair, of course, goes on to become a great success. “Over the next two years… sales start to go up [and this is] one of the greatest-selling chairs in the history of office chairs. That’s a story that tells us something very sobering about the institution of market research. And in particular the efficacy and usefulness of focus groups. Because the whole point of focus groups is to be able to help us predict what’s going to work and what’s not. If a focus group cannot do that, then a focus group is actually useless.”

Really? Whoever said that the role of focus groups is to predict the future? As Gladwell himself reports, the chair went on to win awards. I have no idea who conducted the focus groups (would someone admit it?) or how they were done. How many focus groups were held? Qualitative is qualitative no matter the number but studies with just two focus groups are especially shaky. What kind of people were respondents? How was the chair presented? Did respondents look at a picture, a prototype or a real model? Did they get to sit in the chair and find out how comfortable it is despite its (then) odd look? Obviously, the closer the research comes to consumers’ real-life experience of a product, the better. Without knowing how well the focus groups were done, it’s still fair to say that the research may have been right about initial consumer reactions to the chair - before it won prizes and received media attention.

One input

Focus groups should only be one input in marketers’ decisions anyway. Of course, hypotheses should be tested quantitatively when major/expensive decisions are involved. If a company strongly believes that a product has great potential with effective consumer education and publicity, it makes sense for it to take the risk of moving ahead.

There are a number of other criticisms of traditional focus groups:

  • Stale, stiff format - Respondents are physically and psychologically uncomfortable at the conference table vs. a living-room setting at a facility, their own homes or offices, or some other place (a store, their car, a bar).
  • Intellectualizing - People respond to direct questions with what they consider to be rational answers; people think in images, not words.
  • Bad respondents - Some people lie about their qualifications to get into FGs, participate far too often, come just because of the money and/or food rather than a real interest in the topic.
  • Limited airtime - Respondents have only a few minutes each. Critics take the total FG time of two hours and divide it by the number of respondents to come up an airtime figure of just minutes per person. There is not enough time to get detailed information on people’s history and buying decisions.
  • Group bias - Dominant respondents highjack FGs, talking too much and influencing others.
  • Not for sensitive subjects - People don’t open up in a group on deeper emotional issues (money, sex, illness, etc.).
  • Acting - Respondents try to please the moderator by saying they like the product/ad being discussed.
  • Advertising isn’t experienced in groups - So it doesn’t make sense to ask people about ads when they’re in a group.

Before addressing these criticisms, let’s talk about why there is such a barrage in the last few years.

  • A mature product - FGs are, to some extent, a victim of their own success. Experiencing “been-there-done-that” boredom, some clients search for change, novelty (and sometimes entertainment). For some research suppliers, this is a business opportunity.
  • Mediocre moderating - Some qualitative research practitioners are not very good or are even downright bad, which results in clients becoming disillusioned with FGs as a whole. For example: moderating that is superficial, largely closed-ended, leading, lackluster, theatrical; moderators who cannot control group dynamics, fail to probe, rigidly march through topic guides, talk more than the respondents, and so on.
  • Backroom misuse - Clients who come to instant conclusions, listen selectively to whatever supports their biases, don’t listen (schmooze with other observers), etc. While this has always been a problem, in some ways it is now worse because of computers. Clients check their e-mails during the groups and write up their versions of respondents’ comments, sending their notes out to the client team (but usually not the moderator).
  • Recruiting issues - Respondents who are what the Qualitative Research Consultants Association (QRCA) and Marketing Research Association (MRA) call “cheaters” (lie about their qualifications, demographics) and repeaters (come too frequently, usually lying about past participation). Clients sometimes report they know someone who falls into one or both of these categories; some say they used to do these things themselves!
  • The bashing bandwagon - The more FGs get negative attention, the more clients (market research and marketing) and research suppliers join in. Politically, it becomes increasingly difficult for researchers to recommend FGs since they are viewed as an out-of-date no-no.

It’s important to differentiate between real and perceived problems with traditional focus groups. Some criticisms are at least partially valid, I believe, but this does not mean the answer should be to “abolish focus groups.”

  • Recruiting - No one knows how extensive the cheater/repeater problem is. In addition to the anecdotal evidence I’ve mentioned, some qualitative researchers, including me, have either had respondents admit to or reveal these abuses or have had bad vibes about fishy respondents. Obviously, the same problems arise in recruiting across methodologies; too-frequent participation in online surveys by a small percentage of people, for example, has been discussed lately. FGs are quite literally in the spotlight, however, with clients seeing respondents firsthand; incentives are higher than for most other types of research, too, sometimes attracting the kinds of people we do not want. The QRCA and MRA are both concerned by the FG recruiting issues. One simple step that has been instituted is that many fieldwork facilities now ask respondents to show their IDs when they check in; they are told during recruiting that this will be required, which may help to scare off some cheaters/repeaters. A variety of other measures are also being studied.

Despite my concern about recruiting issues (I’ve been a member of the QRCA field committee for years), my overall sense is that we can trust most FG respondents. In research using recruiting from fieldwork facility databases compared to client lists, mainly people who have never been in a FG before, differences usually do not stand out between the types of participants or the information they provide. (It is easier, however, to recruit upscale people with a list, especially if the client’s name can be mentioned.) Further, hypotheses generated in FGs often do hold up in quantitative research, which shows that the qualitative sample was not oddly skewed.

  • Quality moderating - Clients who choose the lowest-priced moderators, viewing qualitative researchers as a commodity, may end up with poorly conducted FGs and then blame the technique. The importance of a high-quality researcher is nowhere more crucial than in qualitative research where so much on-the-spot judgment is needed; sometimes, but not always, getting quality means paying more. In any case, choosing a qualitative research consultant based only/mainly on price or opting to use an in-house person at the client company to moderate (“How hard can it be?”) can result in mediocre to bad research.
  • In-depth responses - Individual interviews, I agree, are better for storytelling about past experiences and decisions than FGs. The airtime formula mentioned above, however, is a poor way of looking at FGs. Body language “talks” too; comments evoking head nods or shakes, group laughter and animation levels, for example, all tell us a great deal. Further, many sensitive subjects can be explored effectively in FGs; respondents sometimes open up just as much - or more - when talking to a group of peers with the same problems as they do to an individual interviewer.

Effective techniques

Turning to what I consider to be perceived problems, many of the objections to traditional FGs can in fact be handled by effective research techniques.

  • Rapport matters most - Many FGs on emotional topics work well in facility conference rooms. The setting is a professional one that respondents respect and is not necessarily intimidating, as some critics claim. At the same time, it’s important that the room be set up to enhance respondent comfort. For example, the table should be the right size for the number of respondents rather than the enormous ones many facilities have today (fortunately, modular ones can be broken down); chairs should be well padded; light refreshments in the room make respondents feel welcomed and more relaxed.

A good moderator with a warm yet professional style is the critical element in putting people at ease. Active listening - genuine interest, non-judgmental acceptance - makes them feel valued and encourages them to reveal more.

Non-traditional living-room arrangements at facilities can be helpful in certain situations, especially small groups of friends. Stagy ones that don’t look especially inviting or scrunch strangers on a couch without a place to put their coffee cup can actually result in awkwardness. Similarly, “girlfriend groups” have pluses and minuses. Friends are, at least initially, more comfortable with one another than a group of strangers, of course; however, they may be careful not to hurt each other’s feelings by disagreeing or may be reluctant to talk frankly about certain topics with people who know their families.

  • Handling group dynamics - Good moderators know a range of techniques for drawing out quieter respondents and for toning down more talkative and opinionated ones. (It’s unrealistic to think that all respondents will speak equal amounts.) A very simple approach many moderators use, for example, is to have respondents write down comments on the subject or materials before the group discussion. Sometimes the respondents clients think are dominating really aren’t; they may talk a lot, but that does not mean the rest of the respondents are swayed - in fact, they may distance themselves from the big talker.
  • Beyond rational answers - Here, too, good moderators use a range of visual and verbal techniques to get below the surface of superficial responses. Projective questions, video and/or written diaries and collages are just a few. Observations can take place within FGs, too, by watching the way people interact with products, ones they are asked to bring in, handle or prepare, for instance. And we can see how they interact with one another, such as when a brand advocate tells others about “this great product/technology you’ve just got to try.” Often what is most effective, however, is the moderator’s relationship with respondents - creating rapport so that people feel comfortable talking about non-rational emotions and decisions, probing in a conversational way rather than using obvious techniques/exercises (“Now, let’s play a game”).

Growing awareness

So, why should focus groups be used today? Perhaps the good thing about all the FG bashing is a growing awareness that there are, in fact, many variations on the way FGs can be done and a number of other qualitative research techniques are worth using. FGs are not - and never were - the one and only qualitative technique. Our approach should be to figure out what’s the right method for the research’s purpose. My own view, as a practitioner who employs a range of methods, is that there is still great value in FGs when they are done right.

  • Connectivity - Interaction brings out more. FGs trigger respondents’ memories and feelings. Talking together, respondents often build on one another’s ideas. This is especially useful for concept development and idea generation.
  • Consumer connection  Clients can see/hear people firsthand. Despite some backroom abuses, this is generally invaluable. FGs are not the only method offering this benefit, but they are often, for reasons below, the most practical way.
  • Convenience - Time efficiency. The same number of respondents can be interviewed far more quickly, and sometimes effectively, in FGs than in individual depth interviews. (From a moderator’s point of view, four hours of FGs is also less tiring than six to eight hours of one-on-ones.)
  • Cost-effective - A lower cost-per-interview than in-person in-depth interviews (IDIs), which require a great deal of moderator and facility time.
  • Clients convening - The opportunity for conversation. Interaction of observers, together in person or through remote viewing, along with the moderator also stimulates new ideas. The FGs represent time officially set aside to think about and discuss a particular issue or project and form a more concentrated approach than a series of phone calls and e-mails.
  • Con brio - More lively than IDIs. It may sound trivial but FGs are more dynamic to do or watch than a full schedule of depth interviews. By definition, of course, there is more interaction and even more humor. (I am not suggesting conducting FGs for their entertainment, needless to say.)

Rightful place

In short, efficiency, dynamism and insights continue to make focus groups a valuable technique. While the bashing is probably far from over, I do hope that both practitioners and clients will recognize focus groups should have a rightful place in qualitative research.