Editor's note: Naomi Henderson is founder and CEO of RIVA Market Research, Inc., a qualitative research and training firm in Bethesda, Maryland.

Clients have recently made requests like these: "Can you get me the final report on this quartet of groups within a week?" "Can I have a topline report on these Monday groups by Wednesday?"

From the client point of view, the request is not unreasonable. They need written reports to support or challenge their own thinking or provide a basis for decision making for the next phase of the project.

As a moderator, I've said "yes" to both questions, agreeing to support the request for documentation so the client . team could start the next round of decision making. Part of my "yes" has been based in wanting to serve the client and in wanting to be perceived as a qualitative consultant that "delivers."

However, I think that neither the consultant nor the client takes into account the true role of the report and what it takes to fill a request for a full report vs. a topline report. The remainder of this article addresses the two types of qualitative reports and the issues that surround the speed at which each report is generated. There's also a request made of clients at the end of this article.

Top-of-mind, not topline

Topline report is really "top-of-mind. " The original word "topline" was borrowed from quantitative reporting and is an inaccurate term for qualitative research. A true top-of-mind report is available within 24, 36, or 48 hours after the last focus group or in-depth interview (IDI), and it is usually based on what the moderator recalls. It is seldom based on listening to the tapes. There aren't enough hours in the 24, 36, or 48 hour time slot allowed to actually listen to what happened in the groups or IDIs, make notes based on that listening, write a cogent report of the key findings and travel back to homebase. The time allocated for top-of-mind reports only allows for what the moderator can remember and what stood out as the most salient points across the series of sessions.

Conversations over the past few months with other moderators have supported this premise and underscored the desire to provide the best in the way of written results. Here are some sample comments from those conversations:

"I don't have time to listen to the tapes; I just jot down those elements that stood out in my memory and try and link them to the study objectives."

"If I know the client wants a topline, I take more notes on the flip chart so I have something concrete to look at when I write my report--I don't have the time to listen to the tapes and do tasks on my other client contracts. This worries me somewhat because I change the way I lead the groups--spending more time writing on the flip chart than at my seat probing answers."

"I want to give my client the best of my thinking and a topline only lets me give the thinking that I can remember. I don't think it's fair to the client but it is all that the deadlines will allow."

While a topline (top-of-mind) report does address the highlights that the moderator remembers, it doesn't allow for the "considered judgment" of the moderator. A moderator has a lot to juggle during a focus group: hearing from all respondents, deflecting dominators and inspiring shy people, keeping the session on the content target, exposing ideas and materials in a timely manner, getting key information collected in the time set aside and attempting to meet multiple objectives from the backroom observers. Listening and remembering what happened in the focus group becomes difficult in the face of all the other tasks that a moderator is doing "live" and in the moment.

Full report

The definition of a full report is one that covers, presents fully and completely, the methodology, procedures, findings and analysis of the data obtained. This type of report may use illustrative quotes or multiple verbatim comments to support analysis.

In some respects, a full qualitative report is similar to a small Master's thesis. It takes disparate data from a series of groups, with different types of people, across different sites and weaves the comments, reactions and events into a single report that documents something that is not easily measured: perceptions, opinions, beliefs, and attitudes (POBAs).

A Master's thesis takes a long time to write because it takes a long time to collect the data, review it, form hypotheses and then write about those findings in a clear and logical manner. By the same token, writing clear and logical findings about POBAs takes time--not as long as a Master's thesis but it does take time. Even though it doesn't take as long, it does use the same skills as those used for a thesis: sifting out what isn't important, illuminating what is important and why and choosing a frame for reporting the data so that the reader has the clearest insight possible.

Typically, moderators use one of the following methods to write up the results of focus groups:

  • Listening to the groups via audio tape and taking notes
  • Having a transcript made and using it as the base for report writing
  • Having an analyst take notes from behind the mirror and co-writing the report with the moderator

While there are variations on these themes, e.g., using a ghost writer, writing from notes taken in the focus group room, etc., the model still boils down to writing a report based on what happened in the focus groups using notes or transcripts as the basis for analysis of data.

"Rapid-write" vs. "rest & write"

Some qualitative consultants like the notion of having a fast turnaround for report writing, arguing that it is better to go into "crunch mode" and start writing right after the groups are over so that the data is fresh and topical. Those writers tend to prefer working with a behind the-mirror analyst partner and producing the draft report while still on the road or at most, a day or two later. These "rapid-write" moderators can easily fill the request of a client for fast turnaround and their clients are willing to pay the rush fee surcharge that accompanies the speed of reporting.

One moderator indicated that she had mastered the skills of producing fast reports by using specific software and producing "word charts" that easily evolved into the overheads for on-site client presentations a few days later. By working from notes and the recent memories of the groups just conducted, she could offer her clients a finished professional document in a few short days. It must be noted, however, that this moderator stands on a base of 25 years in the advertising research industry!

Other qualitative consultants feel that they need time to carefully review the data from the groups, listening to tapes or reading transcripts and "teasing out" nuances, carefully separating the writer's subjective viewpoint from what was said or done in the groups. The subjective viewpoint re-enters at the analysis stage, and the consultant reviews the findings and measures them against the study objectives. These types of authors could be called "rest and write," to distinguish them from the "rapid- write" types.

Benefits/drawbacks of "rapid write" vs. "rest & write"

One of the benefits of the "rapid write" approach is that the reports are quickly done and the consultant is ready to move on to something new. The drawback is that the consultant cannot do back-to-back projects for a different client in a four day period without paying a high cost in sleep loss. The consultant also risks the chance of missing key client contact support opportunities for the second client of the week. They also cannot spend any time in marketing new opportunities because their attention must be centered on the "work at hand."

The benefit of the "rest and write" method is that it allows for review and rewrite opportunities to look at the data generated and to report data across a broader band of issues. The drawback is that the "rest and write" method requires time - ideally 15 business days from the date of the last group or IDIs. These 1S days are spent as follows: five days for preparation of transcripts or note taking via audio cassette playback; five days for developing themes, outlines and key analysis points; and three to five days for editing and production. The best writing courses indicate that it is advantageous to let writing "rest" a day or two before final edit so that one brings "new eyes" to the reading and supports the development of the clarity that is needed to see it from the reader's viewpoint.

In the qualitative research arena, it has become standard for clients to request rapid turnaround on reports and the rationale given is that it is needed for decision making. However, I question the request. If clients want data on which to base a decision, don't they want the best analysis possible--not the fastest?

I've seen a number of situations with clients where project dates have slipped because more time was needed to revise a concept, or a sample product, or a new advertising campaign approach. Time was allowed for those changes so that what was tested was appropriate. However, the qualitative researcher is asked to "make up" the lost up-front time by speeding up the research report or to provide a report that is "top-of-mind" rather than carefully considered.

Open request to clients

I want to make an open request of clients and qualitative researchers to step back and ask: "Is faster better?" With the increasing support of fax machines, overnight couriers and laptop computers, we've all seen an increase in the speed at which paper is moved through the American business world.

This article is a request to slow down and look at the original purpose of reports and to consider whether this area is one where speed should not have the power that it holds elsewhere. I'd like to see clients resist the "top-of-mind" reports and allow qualitative researchers 10-15 days of time to truly review the findings and analyze the data from qualitative research. I'm asking for the time to write reports that allow the benefits of objective viewpoints from non-vested researchers.

If a report has the power to support decision makers, should it be written mostly from memory by someone with little sleep and limited review opportunities?