Editor's note: Naomi R. Henderson is CEO of RIVA Market Research, Bethesda, Md.

In the soft brightness of fluorescent lights, eight respondents wait earnestly for the focus group session on catalog shopping to begin. Each of the eight receives more than 10 catalogs a month and spends more than $100 each month on items from those catalogs. The moderator gives a clear statement of purpose: "We're here tonight to talk about catalog shopping in general and to look at an idea for a new catalog." General guidelines for participation are given and disclosures are made about taping and one-way mirrors. Respondents introduce themselves, and the moderator easily builds a genial, warm rapport with the respondents.

The moderator asks the first question: Why do you shop from catalogs?

One participant answers, "Because it is convenient."

How is it convenient?

"It saves time - time you would spend driving to the mall."

Another respondent says, "Because there are more choices."

More choices than what?

"Than what you can find in the mall or in department stores."

"I just like the idea of having the world's goods just a phone call away and the books themselves are fun to look at!"

Fun, how?

"You know, you get a cup of good coffee, look at your catalogs, and that's a form of entertainment."

On the surface, the question and follow-ups produce responses that help achieve the study purpose. In addition, the original question meets qualitative research standards of asking easy, non threatening questions at the outset of a focus group.

However, a series of questions proceeding along the same line will set a tone of inquisition, and the focus group will soon fall into an "I ask, you answer" pattern, rather than one that allows respondents to interact. The hallmark of a good focus group is respondents talking to each other and not just responding to the moderator.

The scenario would develop differently if the moderator began by asking: What role do catalogs and catalog shopping play in your life?

"I can't wait to get home to see what new ones have come, I love looking at all the items and marking the pages. For me it is a wish book that I can use to make my wishes come true."

Another respondent takes off on a tangent: "While I like catalogs, I'm feeling inundated these days. If you order something, they put your name on other lists and then you get these strange catalogs with items you would probably never buy."

Yet another respondent takes a different tack: "Yeah, but sometimes you get to see some catalogs for things that you would never see otherwise."

The respondent who first spoke up says, "That's the thing about catalogs. They are convenient and you get all these wonderful options, but some-times you can have too much of a good thing."

Another respondent says, "They are an important part of my life. Talk about convenient! I can't get out to the malls as easily as I used to. With two kids under 6, catalogs are my salvation. I can give great gifts that take only minutes to choose and I get some unusual things that you can't find at the mall. Can I give a complaint?"

The moderator gives the respondent the go-ahead.

"They really soak you on the shipping and handling."

The other participants nod and voice agreement.

"Can't they figure a way to make that less costly?"

The moderator takes a cue from the respondent and opens the question up for discussion: Any suggestions about how a catalog company can lower the shipping and handling costs without absorbing a loss?

Conversation on the topic ensues - the moderator intended to bring up the issue later, but since it emerged spontaneously, the moderator pursues it.

Analyzing the depth of the answers to the two different initial questions, it is clear that the "why" question elicited a paucity of responses and the "role" question produced a waterfall, with many opportunities for respondents to provide rich detail for clients. The "why" question invites a rational, not behavioral, answer; one that begins with "because." A question that starts, "What is the role of . . . " allows respondents to enter the "answer arena" from a number of different directions.

Given the constraints of focus group research (two-hour time frames, the need for relatively equal airtime for responses, multiple client issues to cover and the time of day most groups are conducted), it is critical that every question in a focus group be an effective question. It is eminently helpful to explore the factors listed below as they relate to devising effective questions for focus groups:

  • where questions fit among the key elements of focus group research;
  • role and purpose of effective questions;
  • question types;
  • drawbacks of poor questions;
  • classic or universal questions.

The importance of questions

Good focus group research requires several key elements:

1. A clear purpose statement.

2. The right respondents.

3. A trained moderator.

4. An appropriate research setting (a safe place for communication).

5. The right questions.

The moderator has some control over the first four elements and total control over the last one, which has the most impact on the success of the session. But many moderators say they struggle to find the right questions. It is possible to ask the right questions of the wrong respondents and still collect some usable data. The right questions can be crafted even if the purpose statement is a little murky. An untrained moderator can make several blunders and, with the right questions, still obtain useful data for the client. A moderator can ask the right questions in the wrong environment and still get some useful information.

However, if the first four items are in place and a moderator asks a number of poor questions, the data collected can be weak, faulty, inaccurate and boring! The devastating impact of the wrong questions is one of thing that makes focus group research difficult to sell to savvy clients. If they have been burned by moderators who ask two hours' worth of poor questions, it is no surprise they don't want to conduct qualitative research or trust the information they receive from it.

Role and purpose of effective questions

Traditional focus groups have four distinct stages: introduction, rapport building, in-depth investigation and closure.

The role of any question in a focus group is to elicit data that help reach the study objectives. Every question may not immediately reach the objective but every question should be on the path toward the primary objective. For example, in the introductory stage the questions asked at the time respondents introduce themselves are not oriented specifically to the study objective, but they do give a snapshot of the lives of respondents. "How old are you?" "Who lives at home with you?" "What are your hobbies?" and "How often do you use X?" are all context-setting questions.

When the moderator moves into lines of questions for the rapport building stage, he or she should be asking questions that set the foundations for the critical issues that need to be explored in the in-depth investigation. Questions in this stage should be easy to answer and allow respondents a chance to flex their answering muscles.

When the session moves into the in-depth investigation stage, the questions tend to become more precise and more specific. In this stage, each question should clearly support the study objectives and ideally build on the other questions. Any number of strong questions could nicely move the catalog project along:

What items are missing from the catalogs you like?

Can you give me some examples of 'unusual' items that you have found only in catalogs ? What are some reasons these items aren't in stores?

If you were starting your own catalog of unusual items for the home and gift giving, what categories of items would you include?

There are no readily available, pre-formed answers for any of the questions above. The answers could go down a number of paths. As long as the questions support the study objectives, they are appropriate. The degree to which each question produces a rich body of data that can be analyzed is the measure of its value.

What's more, the questions don't provide part of the answer. A true question is one to which you don't already know the answer. Focus group research should find answers we don't already know. The best way to get to those answers is by asking clear, specific and precise questions that will elicit a broad range of answers.

Questions for the closure stage are typically general in nature and are meant to close down the conversation:

Is this focus group discussion what you expected?

Did you learn anything new about catalog shopping by being here tonight?

Question types

There are many categories of questions (intrusive, eliciting, open-ended, challenging) and an entire article could be written about them. But there are primary categories that form the basis for most qualitative research: general, specific and probing.

The lore of the industry indicates that it seems to work best to move from the general to the specific when asking lines of questions within any one issue area, and to probe when more data is needed. That sounds so simple. So how can a researcher make so many mistakes when moving from general to specific, then probing for more detail? He or she does not fully understand the nature of general, specific and probing questions.

In the shortened time frames in which most projects have to be completed, moderators often don't know what issues a focus group has to cover until a few days before the session.

So, under a great deal of pressure, they write either an outline format and create the questions during the group, or they write a detailed guide with anywhere from 25 to 75-plus questions. When questions are constructed in the focus group, a moderator runs the risk of having to form questions out loud and needing to track and backtrack until the right question gets asked. The process is hard on the moderator and makes it difficult for respondents to focus on their answers. It's hard for the client as well, since the great question that got asked in the first group may not be remembered and asked again in the second because it wasn't written down.

Problems can arise when the questions are written out in some detail before the group but the moderator lacks sufficient time to properly organize them. When they are written out beforehand in shorthand, they are not as effective as fully formed questions. Having seen the guides used by moderators who have been in business for a number of years, I'm convinced there is a macro on their computers that allows them to easily add "Why/ why not?" to a specific question or statement.

I've seen that phrase in places where it did not belong and I know it is a reminder to the moderator to probe. However, under the pressure of leading a focus group, the mind will access any piece of data it can find to advance the process, and when it sees "Why/why not?" that is what it tends to ask! Doing so limits the number of paths the focus group can explore rather than leading to new areas of exploration.

I once gave a client a draft guide to review so they could see the question paths I intended to take. The client called back and said, "We are fine with the questions and the logic path and we want you to add a section on ad recall before you show the new ads. Also, we noticed that there aren't any 'why' questions in your guide. Why is that?" I had to suppress a great deal of laughter before I could tactfully say: "I tend to get richer data if I avoid the word 'why' and ask respondents for reasons or examples. Let's talk about 'why' questions after the first pair of groups and see if any should be added back in. " As it turned out, the client forgot about "why" questions because staffers were too busy sorting out all the new insights they got from the focus groups!

General questions open up an issue so that a base is established:

What are some of the reasons for so many catalogs in this country?

How do you think you get on so many mailing lists for catalogs?

Specific questions elicit more than top of mind answers. These specific questions stand on the shoulders of the general questions, and they should flow logically:

Here's a new idea for a catalog. [Show sample pages.] In what ways are the items on these pages like all the other catalogs you see?

Can you tell me any items you see here that are truly unique? Which ones are not typically in stores or other catalogs?

What do you need to know about the country of origin of products in a catalog like this new one?

What do you need to know about these items to accept or reject these catalog offerings?

Probing questions unearth more information about a specific point made in response to a previous question. The trick is knowing when to probe and when to move on. My ground rule for probing is when I think I know what a respondent means I ask one more probing question so they can confirm that point aloud for me. I'm often surprised that what I thought I understood (based on my own internal reference) is not confirmed by the respondent. He or she actually sees it in an entirely different way.

When a participant gives an answer that includes a word for which various people have various interpretations, I probe to discover his or her specific reference. Words that fit this category include: nice, pleasant, fine, appropriate, convenient, sexy, fair and in-style. If these words or their like are liberally sprinkled through a respondent's answer, I find the one that I need to understand more clearly and probe:

What exactly do you mean by saying catalog shopping is 'convenient' - how, in what way is it 'convenient?'

Can you give me an example of what 'in-style' means to you.

If you are buying a nice gift from the catalog what evidence would l see as an observer?

Drawbacks of poor questions

Poor questions exact a price, sometimes a very dear one, on the research process. The research can suffer in a number of ways:

  • study objectives not realized;
  • respondents focus too closely on the question process and not their perceptions, opinions, beliefs and attitudes;
  • respondents get bored;
  • respondents talk, but don't really answer the questions;
  • client sends in lots of notes to attempt to focus the lines of questions;
  • moderator under a lot of stress and must "pull teeth" to get data out of respondents;
  • clients feel their needs were not served;
  • qualitative research gets a bad name.

The classic 10

Some questions seem to work no matter the topic or situation. Universal or classic questions are not magical, they are just useful questions that open up areas for discussion:

1. If you were in charge, what kind of changes would you make?

2. What would it take for this [product, service, ad, concept, idea] to get a gold star? Or, if this [product, service, etc.] received an award, what would it be for?

3. If you were the moderator, what would be the next question you would ask the group?

4. What would you tell a best friend or family member about this [product, service, etc.]?

5. Assume this [product, service, etc.] could talk, what would it say about itself?

6. If you could only change one thing about this [product, service, etc.] what would you change, and what's the main reason that one thing needs changing?

7. After respondents have graded an item, and they have given it something less than an A, ask, What would it take for this to get an A?

8. When respondents clearly don't like something and they are making a lot of very negative statements, shift the group's attention to its positive aspects by asking, Can you tell me five positive things about this [product, service, etc.], no matter how small that positive thing is?

9. If you were responsible for selling 1,000 units of this product, what key point would you stress in the ad campaign?

10. What do you need to know about this [product, service, etc.] in order to accept or reject it?

Again, there is nothing magical or special about these questions--they just seem to open up new lines of conversation. If a reader has any, questions like this that fit a universal or classic theme I would love a letter from you so I can add them to my list. (RIVA Market Research, 4800 Montgomery Lane, Ste. 1000, Bethesda, MD, 20814.)