Rethinking focus group reporting: dynamic debriefing

How to wean yourself from written focus groups reports that take too long, cost too much and don’t go far enough

Editor’s note: Richard Feder is president of The Marketing Group, Inc., Stamford, Conn. Bryan Mattimore is president of The Mattimore Group, Stamford, Conn.

With the "full speed ahead" pace of business today, companies just can’t wait for a detailed written report on focus groups. Reviewing the tapes or transcripts from four focus groups, writing down findings, organizing them, developing conclusions/hypotheses and writing a deck takes an experienced moderator/analyst around 20 hours. This adds weeks to the project timetable and up to $6,000 to the bill. Both the time and the money are unaffordable.

What can you do?

Quick and dirty are high-risk alternatives. An informal debriefing after the last group or a top-of-mind topline run the risk of just skimming the surface - missing a big opportunity or making a big mistake. There’s got to be a better way. There is. It’s called dynamic debriefing, and more and more companies are turning to it.

Underpinnings

Dynamic debriefing sessions are usually conducted the day following the last group and take five hours. What makes the sessions dynamic is not just their speed, but also the concentrated intelligence they bring to bear on the business implications of the new learning. Roughly 60 percent of the time spent preparing a traditional focus group report goes into identification of findings and development of conclusions. Writing takes another 20 percent. That leaves only 20 percent of the time to figure out what it all means and what you should do about the findings. And this is 20 percent of the time (or about four hours) of just one person representing only one perspective.

Dynamic debriefing allocates 70 percent of the time of six to 12 observers plus the moderator/facilitator (25 to 45 total hours) functioning at the height of their creativity to the crucial action issues. The process that makes this possible combines:

• a simple, disciplined listening system to help observers develop findings and conclusions during and between groups;
• facilitated discovery techniques to rapidly identify and organize key findings and conclusions;
• small team ideation exercises to explore ramifications of key conclusions and to formulate and refine action plans;
• resources to produce a transcript of findings, conclusions and recommendations as they are developed (so you leave a dynamic debriefing session with your written report in hand).

What do you give up to get this instant turnaround? Nothing. The document you leave with is actually better than a traditional focus group report. It contains plenty of quotes, all the conclusions you could ask for, plus the best thinking of more than one individual focused on "What next?" rather than "What happened?"

Data development

The dynamic debriefing process requires a focus group moderator/session leader skilled in interviewing, analysis, facilitation and development of successful business strategies. You also need a "technographer," someone to record and organize output from the debriefing session while it occurs. But neither of these people is as important as the six to 12 observers who identify the findings and develop the conclusions on which the debriefing is based. Their job is not that difficult, but it does require marketing acumen and a fairly high degree of dedication.

During the groups the observers record on the left side of a page all respondent comments that sound like they might have important impact on what the target group buys, uses or thinks about the research subject. After each group the observers review their notes and jot down on the right side of each page the conclusions these comments would suggest. This requires some extra work, but it has the positive effect of cutting down on distractions in the back room and reducing M&M consumption (which can be a major element of research costs).

Between the groups and before the debriefing, the observers review their notes and identify the conclusions they feel have the most important business implications. These conclusions and the respondents’ comments on which they are based are the building blocks of the dynamic debriefing session.

Data processing

The day after the last group, the observers arrive at the debriefing session armed with pages of notes on what respondents said and with their hunches about what these comments mean. (Usually, they also arrive bleary-eyed, but a little fatigue is a small price to pay for the clearer recall and extra action time you get from debriefing immediately.)

Dynamic Debriefing Session Plan

1. (30 min.) Introduction

• Review of research objectives
• Review of business concerns prompting research
• Review of other relevant information
• Identification of important facts/findings not covered by groups.

2. (60 min.) Findings and conclusions.

• Individual development and posting on easel pad sheets of the most important conclusions
• Development of consensus list of conclusions and identification of supporting findings
•  Discussion of additional important conclusions
• Organization of conclusions by subject
• Clustering of conclusions in each subject area

3. (10 min.) Break

In a dynamic debriefing, development of findings and conclusions, and separation of important ones from those which are merely interesting, is accomplished via an application of the 80/20 rule. The 80/20 rule is based on the observation that 20 percent of products usually account for about 80 percent of the sales; 20 percent of users usually represent about 80 percent of consumption, etc. (This principle is sometimes referred to as Ferkhoff’s Law. Gene Ferkhoff was a pioneer in discount retailing. When asked by the Harvard Business Review to describe his inventory control policy, he replied, "Heavy on the best and the hell with the rest.")

In this application of Ferkhoff’s Law, participants start with a background review by the brand or research manager of the situation prompting the research, the research objectives and any relevant information not covered in the groups. Against this backdrop each participant selects from his or her notes the 15 to 20 conclusions they think have the most potential relevance and importance to the problem at hand and record them on an easel pad sheet. The group then stands back, ignores differences in wording and identifies the conclusions that appear on many people’s sheets. While the facilitator records these consensus conclusions on index cards and posts them on a wall, the comments/findings on which each conclusion is based are reviewed (and recorded by the technographer). At this point any participant with a strongly held belief in the relevance and importance of any other conclusion can make a case for adding it to the short list by citing the comments on which it is based and the business implications it might have.

Once the final list of consensus and impassioned conclusions is posted, participants work together to organize the conclusions by subject (e.g., distribution, awareness, etc.) and then to create (and label) clusters of related conclusions under each subject.

By now, only 30 percent of the session is in the past tense and all the key findings and conclusions have been developed and organized. This is one of the most exciting parts in the debriefing session, as participants step back and see how much they have learned and how it all hangs together. In fact, it’s such a high that everybody needs a break. So take one.

Using the database

You don’t conduct focus groups just because you want to know how people think and feel. You conduct them to help decide what to do about how people think and feel. Therefore, the largest portion of dynamic debriefing time is devoted to formulation and refinement of action recommendations.

4. (90 min.) Tentative recommendations

• Individual selection of conclusion cluster
• Formation of pairs
• Pairs action ideation for each conclusion cluster
• Pairs selection and boarding on easel sheets of highest potential action recommendations for each cluster
• Brainwalking: pairs build on one idea for another pair; go to new sheet and repeat (until each easel sheet has been built on by each pair)
• Sheet owner selection of highest potential actions
• Pairs presentations

5. (10 min.) Break

To start, each participant signs up for one cluster of conclusions on which to work. (No duplications allowed.) Participants then pair up to work together. Wherever possible, people with different backgrounds are teamed together (e.g., a brand manager with someone from R&D, etc.). Each pair then uses each of their two conclusion clusters as a stimulus to come up with as many action ideas (i.e., tentative recommendations) as they can. After about 10 minutes on each cluster, they stop, identify the one or two action ideas they judge to have the greatest impact potential for that cluster and post each on a clean easel sheet.

The next step in the action development process always works and never fails to amaze. It’s called brainwalking. Brainwalking is a seemingly simple process that frequently produces insights which border on brilliance. It starts with each pair of participants in front of an easel sheet from another pair. The pair then builds on the idea or comes up with a new idea and writes it down on the easel sheet which sparked the idea. This process is repeated until every sheet has been built on five or six times (which is the number of iterations usually necessary to get from minor modifications to out-of-the-box ideas, if there are going to be any). Each pair then returns to their original easel sheets, picks the single action suggestion on each of their original sheets that they think has the greatest impact potential and presents their recommendations to the group.

This is another of the emotional high points in a dynamic debriefing session. Participants are now surrounded by actions the company could undertake, some with low risk and modest potential, some with almost atomic potential and high danger of fallout, and a range of stuff in between. This is the ideal time to stop and pat each other on the back, which requires another short break.

Gem polishing

Companies have a finite capacity for action. The next step in the process is identification of those tentative recommendations which most merit enactment.

6. (100 min.) Refine recommendations

• Criteria development
• Dot voting identification of highest potential actions
• Individual sign-up for actions to champion
• Formation of new pairs
• Pairs ideation of consumer (or customer) advantages/benefits of actions
• Selection of key advantages/benefits * Pairs identification of primary corporate concerns with actions
• Modification of actions to overcome concerns
• Presentations of refined recommendations
  
  (Repeat for additional actions with "Great Thinkers")

Prioritization of recommendations

To help participants decide which of the tentative action recommendations are most appropriate to the situation and the company, review the project objectives and develop action criteria. With this as a backdrop, a fast and fun way to accomplish the winnowing process is with dot voting. Each participant is given eight to ten adhesive dots and told to allocate them to the tentative recommendations according to their best judgment of fit with the criteria. Counting up the dots each action idea garners separates the "best" from the "rest."

The process of making these "best" even better starts with each individual picking one of the surviving tentative recommendations to champion. (Again, duplication is not allowed.) Then form new pairs (for variety). Recommendation refinement starts with each pair focusing on one of their tentative action ideas and listing as many potential buyer advantages and benefits as they can. They then star the one or two most important. This activity increases enthusiasm for the idea and sets some priorities about which buyer effects of the tentative recommendation are most leverageable and least expendable. Against this backdrop each pair addresses the question of "What are your primary concerns or fears about the company undertaking this action?" After each pair has worked in this negative, but necessary, vein for a while, their efforts are directed to "How might you modify or restructure the action to overcome each of these risks?" After about 15 minutes on each tentative action, each pair presents their refined recommendations (usually to deafening applause).

The next step is to repeat a variation of this exercise to cover the other tentative recommendation being championed by each pair. When people repeat an exercise exactly, they work faster (because of the learning curve), but their effort usually produces less spectacular output (because of the "been there, done that" effect). Therefore, it’s best to interject some new element into the second pass. In this case, a good "new element" for the second pass (to insert after identification of concerns) is "Great Thinkers." Each participant selects a personal consultant from a list of people like Walt Disney, Eleanor Roosevelt, Leonardo DaVinci, Bill Gates, etc. After making their selection, they are each given a card which provides a brief bio, a list of the person’s key accomplishments and a summary of their strategy or philosophy. Each participant "asks" their Great Thinker for a fresh perspective on the concerns and for some advice on how to approach them, writing down what they think Leonardo or Eleanor, for example, would suggest. Each member of the pair then reads the advice to their partner and together they use their priceless consultants’ good counsel as a stimulus to develop idea modifications to deal with their concerns.

The final step after the presentation of the last round of refined recommendations is the determination by participants of the relative attractiveness to the company of each refined recommendation and the assignment to each of a relative priority.

Going home

We developed dynamic debriefing to solve the business problem of how to get more useful information from focus groups, faster and less expensively. We never reckoned that it would also be highly enjoyable. Facilitated correctly, a dynamic debriefing session is a heady experience. One day after the groups, you leave with a feeling of having been part of an exceedingly successful team effort. You also leave with a transcript concisely documenting where you ended up and how you got there. The transcript starts with a series of well-reasoned recommendations based on 25 to 45 hours of time from six to 12 impassioned people - more than ever went into the recommendations section of any traditional written focus group report. And some of the recommendations probably border on sheer genius. (After all, they were produced by a diverse group of knowledgeable, committed people working together on creative exercises that let them build on each other’s best thinking.) Right behind the recommendations in the transcript are the conclusions, each supported by the findings and quotes on which it was based.

And you can review this transcript on the trip from the group site back to the office, weeks before you would have gotten a written report that would have given you recommendations formulated with substantially less brainpower.

That’s what’s dynamic about this debriefing.