How focus groups can help questionnaire development

Define and refine

Editor’s note: Mark Davies is lecturer in marketing at Heriot-Watts University, Edinburgh, Scotland. Mel Prince is professor of marketing at Southern Connecticut State University, New Haven, Conn.

The use of focus groups for questionnaire development is well known and widely touted. Most marketing researchers are keenly aware that focus groups may improve the quality of survey instruments. However, few marketing researchers have a full appreciation of applications where focus groups are most fruitful, the range of benefits that accrue, and the special qualitative methodologies that may be employed to maximize these benefits. Thus, general codification of the methodology of focus groups for questionnaire design is an urgent need for marketing researchers and their clients. This article examines how protocols, developed through focus groups, can be used to substantially improve survey questionnaires.

The overall aim of using focus groups to aid in questionnaire design is to improve the quality and validity of information obtained in surveys. As a means of achieving these aims, focus groups provide insights and understanding of study subject matter, can help develop hypotheses for the quantitative study, discover new content areas, refine and classify survey content, and eliminate meaningless questions from the final questionnaire. It is useful to enumerate the range of benefits of qualitative analysis for questionnaire design. However, it is even more important to develop guidelines to achieve what is desired.

This article codifies situations and methods to show how focus group research can be used for questionnaire development. Much is known about basic criteria of suitable questionnaire design. What is not generally known is how focus groups may be used to determine appropriate question wording, issues to explore and the structure of scales that are employed. Questions determined from practitioner intuition or analogous experiences are extremely risky as they do not represent actual user feelings and opinions, attitudes and behavior.

Why focus groups are useful for questionnaire design

Focus groups are especially useful in questionnaire design for each of the following problems associated with questionnaire design: finding more effective segmentation criteria, eliciting emotional motives, learning consumer language for communications programs (Gordon and Langmaid, 1988: 3-5; McNeill, Sanders and Civille, 2000), and screening measurement areas. In sections to follow, we discuss each of these problems and provide some illustrations of techniques that may be employed. It should be noted the techniques that are illustrated may be applicable to a variety of questionnaire design problems.

A. Segmentation criteria

There is an increasing need to segment more precisely in consumer research since today’s consumers exhibit a varied, complex and sometimes conflicting set of preferences. New sources of segmentation need to move beyond conventional demographics and psychographics.

Supportive techniques to develop segmentation criteria include the use of psychological projection. Templeton (1987: 249-250) has shown how figure drawings might be used in discovering the personalities of users and non-users for a discussion guide. Respondents to a discussion group might be asked to draw a figure of a person who they feel would subscribe to a particular magazine, purchase a candy bar, or join a particular charity. They are later asked to draw a figure of a person who would not use these products.

For benefit segmentation, in particular, Wade (1998) supports the use of focus groups in separating the characteristics of a product from the benefits that have obvious utility in communications programs. Wade (1998) recommends comparing and contrasting questions using side-by-side listings to uncover dimensions and attributes. Personal experience stories can describe how characteristics deliver benefits and why they are important.

If the objective is to reposition a brand, debate groups involving groups advocating rival brands may be quite useful. Another outcome of the brand debate approach is that the list of attributes and benefits will be reduced to those most salient to brand acceptance or rejection.

B. Emotional motives

It is not obvious or easy to penetrate the emotional side of behavior. First, it requires a level of understanding far deeper than that which can be gained from merely examining previous purchase behavior. Second, low-involvement items (routine, low-ticket items) are unlikely to be subjected to personal scrutiny each time a purchase is made, so the researcher must consider ways to examine purchasing motives from alternative perspectives to direct questioning (Templeton, 1987:16-17).

Psychological projective techniques that adopt oblique questioning have been used to uncover emotional motives in focus groups. Projective techniques have been used to encourage consumer openness about their own motives and feelings, beliefs and actions, and so tend to encourage more negative responses. With a low-involvement item, the moderator might place the product in a strange context, or ask how other people might use the product. Templeton (1987: 18) suggests the use of a fantasy trip in soliciting thoughts about essential cosmetics required to live in an exotic location.

Alternatively, respondents can be encouraged to shift their way of thinking more directly. Templeton (1987: 246) shows how the emotional side can be tapped into, using magazine subscriber profiling, as in this example:

“You’ve been very logical and articulate so far. Right now, I’d like you to turn off the logical part of your heads for a moment, and deal with some illogical, feeling-type distinctions. I’m going to pick some of the magazines and papers you singled out for mention, and ask you to imagine that each one is a person, whom you are trying to describe to somebody. Tell me about the (e.g., Ad Age) person. How old is he or she? Is it he or she? Married or single? What kind of car does s/he drive? Overweight or thin? Conservative or vogue-y? How about politics? Serious and buttoned-up or more freewheeling?”

Thus respondents can be encouraged to change from rational thinking to emotional feeling within the group discussions.

Focus group research may be used to identify the hidden drama in products that appear the most mundane on the surface. Reification of a brand can reveal the relative strengths and weaknesses from an emotional perspective. Describing a brand’s epitaph, or the houses in which a brand might live, etc., achieves a kind of personification, creating the kind of refined imagery necessary.

C. Consumer language

Face-to-face dialogue with consumers in focus groups elicits consumer language about marketable objects. The advantage is that nuances of meaning can be identified from slang and from words that share a diffuse imagery (for example, words that share similar meanings that could easily be used out of context). The moderator can probe beneath the mind’s surface for the meanings of words. For example, the word “crunchy” applied to a snack bar might indicate crispness to one group of consumers (suggesting positive associations) whereas to another it might indicate “hard to digest” (negative associations). Consumer language suggests expectations (e.g., this must have a biscuit base, or a nutty core) or reveals how the product is accepted by others if consumed in public (e.g., makes a crackly, embarrassing noise when bitten, implying negative associations). The number of identifiable and distinctive positive and negative associations can be useful for designing questions that will elicit product imagery in a survey.

Identifying consumer language is also useful for refining fundamental research questions and for identifying specific indicators of constructs such as brand loyalty that may otherwise be unknown when exploring new research areas.

In addition, focus groups can be used to improve reliability by helping better match response scales to how respondents actually evaluate (Wade, 1998). McNeill et al. (2000) reveal how this can be done, in refining attribute scales of peanut butter using generic descriptors of peanut butter taken from secondary research based on the peanut lexicon (Johnsen et al., 1988) and the semisolid texture terminology (Meilgaard et al., 1991). These generic descriptors were: appearance, flavor, and texture of peanut butter. Participants were asked to describe the peanut butter for each of these generic descriptors to identify specific descriptors and their direction (positive or negative). Consumers were also encouraged to describe products based on their own terms, beyond those merely associated with attitudes and usage. Consumers offered additional non-attribute-driven purchase influences to those reported by the peanut butter lexicon and semisolid texture terminology. These included early brand preferences, price, jar size, and reason for use (direct, or used in home baking) as strong influences of brand loyalty (McNeill and Sanders, 2000: 177).

Had no existing research been conducted on descriptors, the researcher would have been reliant on intuition and experience in conducting focus groups for products close to, but not identical to, peanut butter to guide their construction of a flexible research guide. Such an application of focus groups in exploratory research can chart unfamiliar territory for the researcher. The interviewing is much more open-ended, revealing variations in perspectives and attitudes.

Focus groups allow respondents to disclose a range of opinions on a set of issues and the circumstances that lead to one or another response (Morgan and Krueger, 1993: 18). Detection of a wide variation in responses to questions would suggest that the level of response options or scales used in questionnaires might need to be broadened. Although this can be identified through individual interviews, focus groups make more efficient use of sponsor time in commissioned research (Schechter, Trunzo and Parsons, 1993). Zeller (1993) suggests using a screener that encourages respondents to mull over topics before the focus groups begin, since this can promote richer discussions.

Since the level of professional and technical terminology is extended with establishment-survey questionnaires, there is an additional need to test for comprehension. Dillman (1978) has supported the use of precise language for respondents who share a particular understanding of the meaning of words by virtue of their training. Precise definitions for problem words identified in focus groups may be located on questionnaires to facilitate consistency in what is measured.

Analyzing non-verbal behavior

Analyzing and interpreting data may require moving beyond the spoken word, since people often do one thing and say another (Griggs, 1987). The moderator, if suitably skilled, can also pick out non-verbal behavior that may be relevant (e.g., gestures) when discussing products. By using neurolinguistic programming, the moderator can identify patterns in eye movements or hand, head and body movements that indicate whether respondents are talking truthfully or simply acting (Johnson, 1993). Transactional analysis is another technique that can be used to identify shifts in ego states (associated with parent, adult and child, for instance, Gordon and Langmaid, 1988: 146-148) regarding marketable objects, which may offer insight into why communications may be accepted or unaccepted.

D. Screening measurement areas

The focus group can be as important for deciding what to exclude from a questionnaire as what to include. Focus groups can act as filtering or screening mechanisms to help narrow the ideas for hypotheses generation that may be tested in questionnaires (O’Brian, 1993). Here, the experience and judgment of the moderator may be critical in determining an inventory of attributes. Attributes selected for a survey should not be entirely dependent on votes or frequency of mention, since these have not always found to be reliable indicators of degree of importance (Abels, Domas and Hahn, 1997). An attribute may be particularly important despite it seldom occurring in practice. Personal experiences might indicate why or how an attribute is important, and offer more information than frequency of mention alone. Indicators of importance also include stress on words, excitement conveyed, interests captured by other members, changes in tone and body language.

Summary

Constructing questionnaires often involves knowledge of criteria used for market segmentation. Further problems include discerning emotional issues that drive consumer behavior, incorporating the language of consumers into questionnaires and screening items for inclusion. Using focus groups can reduce these problems in questionnaire design. In exploratory research, focus groups are particularly productive in establishing a broad range of marketing issues beyond consumer attitudes and usage. Once the broad range of issues has been identified, further discussion can establish the boundaries of these issues. The results are a precise focus on issues that are critical to the inquiry.

Focus groups are especially useful for establishing quickly (and efficiently) the range of issues that determine attitudes, opinions and behavior. Speed (and cost-effectiveness) is based on the cross-fertilization of ideas allowed in focus groups. This information enables an optimal battery of attributes and range of response options to be identified. This includes appropriate scale semantics and metrics. Intervention techniques involve projective methods and skilled probing.

Focus groups allow an experienced moderator to check on the general strength of feeling based on intensity of agreement and through signals of non-verbal behavior.

More work needs to be done to determine the kinds of research problems and instruments where focus groups can be most productive and the specific approaches that work best under these varying conditions. Among the important considerations are the samples used in focus groups, the character of the interventions, and the methods of analysis that are employed. Sample strategies include the use of homogeneous samples to refine perceptions, or adversary, interactive samples that argue different positions, or diverse small groups that are cohesive in problem solving. Suggested projective techniques include personal experience stories, brand personification, role-play and reification, and compare-and-contrast questioning techniques. | Q

References

Abels, E.G., White, M.D., and Hahn, K. (1997), Identifying User-Based Criteria For Web Pages, Internet Research: Electronic Networking Applications and Policy, 7, 4, 252-262.

Bureau, Marcel (1991), Experience With The Use of Cognitive Methods in Designing Business Survey Questionnaires, Proceedings of Survey Methods, American Statistical Association, Washington D.C., 713-717.

Callingham, M. and Baker, T. (2002), We Know What they Think, But Do We Know What They Do?, International Journal of Market Research, 44, 3, 299-335.

Diamantopoulis, A., and Schegelmilch, B.B., Determinants of industrial Mail Survey Response: A Survey-on-Surveys Analysis of Researchers’ and Managers’ Views, Journal of Marketing Management, 12, 505-531.

Dillman, D. (1978), Mail and Telephone Surveys: The Total Design Method, New York: John Wiley.

Gordon, W. and Langmaid, R. (1988), Qualitative Market Research, A Practitioner’s and Buyer’s Guide, Gower: Aldershot, UK.

Griggs, S. (1987), Analysing Qualitative Data, Journal of the Market Research Society, 29, 1, 15-34.

Johnsen, P.B., Civille, G.V., Vercellotti, J.R., Sanders, T.H., and Dus, C.A.Q. (1988), Development of a Lexicon For The Description of Peanut Flavour. Journal of Sensory Studies, 3, 9-17.

Johnson, K.J. (1993), Selling with NLP, Nicholas Brealey, London.

McDaniel, C. (1979), Focus Groups-Their Role in the Marketing Research Process, Acron Business and Economic Review, 10, 14-19.

McNeill, K.L., Sanders, T.H., and Civille, G.V. (2000), Using Focus Groups To Develop A Quantitative Consumer Questionnaire for Peanut Butter, Journal of Sensory Studies, 15, 163-178.

Meilgaard, M., Civille, G.V., and Carr, B.T., (1991), Spectrum terminology for descriptive analysis. In Sensory Evaluation Techniques, Second Edition, 162-172, CRC Press: Boca Raton, Florida.

Miles, M.B. and Huberman, A.M., (1994), Qualitative Data Analysis, Sage: London.

Morgan, D.L. and Krueger, R.A. (1993), When to Use Focus Groups and Why, In Successful Focus Groups, Ed. D.L. Morgan, pages 3-19, Sage: London.

Myers, J.H. and Shocker, A. D., (1978), Toward A Taxonomy of Product Attributes, Working Paper. (Los Angeles, University of Southern California, June).

Robson, S. and Hedges, A., (1993) Analysis and Interpretation of Qualitative findings: Report of the MRS Qualitative Interest Group, Journal of the Market Research Society, 35, 1, 23-35.

Schechter, S., Trunzo, D., and Parsons, P.E. (1993), Utilizing Focus Groups in The Final Stages of Questionnaire Design, Proceedings of the Survey Research Section of the American Statistical Association, Washington, ASA, 1148-1153.

Scott, P. (1999, May). Inspiring Creativity With Better Briefs and Briefings. Presentation given at the Account Planning Group, U.S. Annual Conference, San Diego, CA.

Templeton, Jane F. (1987), Focus Groups: A Guide for Marketing and Advertising Professionals, Probus: Chicago.

Thomas, J.W. (1998), Finding Unspoken Reasons for Consumers’ Choices, Marketing News, 32, 12, 10.

Tomaskovic-Devey, D., Leiter J., and Thompson, S. (1994), Organisational Survey Non-Response, Administrative Science Quarterly, 39, 439-457.

Wade, R.K. (1998), Focus Groups Can Help Develop Survey Issues, Marketing News, 32, 5, 18.

Walker Research Inc. (1992), Walker Image Study, Research on Research. Indianapolis.

Zeller, R.A. (1993), Focus Group Research on Sensitive Topics, In Successful Focus Groups, Ed. D.L. Morgan, pages 167-183, Sage: London.