Guidelines for success
Editor’s note: James P. Murphy is an independent marketing and opinion research consultant based in Princeton, N.J.
New products, new advertising campaigns, new and relaunched brands, and new corporate and institutional entities require consumer research to support decisions affecting their marketing communications programs. Researchers play a vital role in this process through the design and implementation of studies yielding results that lead to effective communications strategies. What follows are guidelines for the design and conduct of such studies that this researcher feels are crucial to their success.
1. Understand the challenge.
What branding concepts, product names, slogans and ad copy have in common is they are not the physical product or service itself. It is easy to ask people if they prefer a four-ounce hamburger or a six-ounce one. They can answer from memory or we can give them samples. We can learn who prefers which and in what proportions. Consumers are comfortable telling us about their purchasing habits when the determinants are objective criteria, including convenience and value, that allow them to sound rational. Branding and similar strategic tasks for the marketer take the respondent into a different realm, however.
Our question is usually, “Would you buy this?” And it is asked after having presented a characterization of the product or service. The answers that respondents give to our question are often affected by assumptions they make about the characterization. For example, is it: a quote from a reputable, objective, third party; a statement associated with the sponsor which itself has some level of credibility (high or low); or a claim lacking either of the above?
The more the characterization is viewed by respondents as “just advertising,” the more they will lower their reported likelihood of purchase in order to avoid making themselves appear to be overly impressionable, credulous or naïve. The content validity (truthfulness) of advertising copy is discounted by consumers because in their eyes it’s “just advertising.” That’s why so much ad copy employs high-credibility endorsers, third-party endorsers or otherwise attempts to present its argument with enough factual backup that the reader can discount the claim only at the cost of living with awareness that he is now calling the advertiser a liar.
Any exercise in this area must be packaged in a manner that is realistic to respondents and therefore allows them to unselfconsciously reveal the expectations they would have about a product described this way or that way, along with the reasons for those expectations.
2. Understand the roles of qualitative vs. quantitative methods.
Qualitative studies - focus groups and individual interviews - yield insights into the hows and whys of consumer behavior, and the likely consequences of alternative strategies. They are diagnostic and exploratory in nature. Quantitative studies (surveys) address the impact of strategies already in place or contemplated. Which execution is more persuasive? Which types of consumers prefer Logo A over Logo B? Important conclusions developed from numerical results of qualitative research that is being used for marketing decisions may need confirmation in subsequent quantitative research using larger and demographically-projectable respondent samples. Conversely, qualitative research is often used to expand our understanding of factors responsible for a pattern of findings in a survey.
3. Get buy-in from all stakeholders, including client, agency writers and designers, and agency account management.
Agree on what kinds of learning can be expected from the effort as well as what kinds of conclusions are not realistic to expect.
4. Encourage observation by those responsible for acting on the findings.
Having observers at focus groups, for example, strengthens the credibility of conclusions presented to senior management and others who were not present. It gives everybody the same frame of reference. There is a better-informed and more challenging audience to whom the researcher must present and defend the conclusions.
5. Test a control.
New material is usually seen as unique and the impulse is to evaluate it on its own terms. When we learn that 60 percent said “definitely would buy,” we feel good. But should we? The best way to answer that question - even in focus groups - is to replicate the procedure with existing material of known effectiveness. If the control scored 40 and we reach 60 with a new execution, we know we have a strong concept. Only this type of yardstick can tell us that. (Note: Test and control materials should be presented in equivalent formats and in some situations it may be necessary to adjust for the fact that respondents are already familiar with the control.)
6. Use consumer language.
An execution is presented and the moderator asks, “How do you feel about this?” Someone behind the mirror mutters, “Jim, I don’t care how she feels about it. I want to know if she will buy it!” What this observer fails to appreciate is that if the respondent had been asked that question, she would feel insulted. “I don’t spend my money on the basis of cartoons somebody shows me in a focus group!” Instead, oblique and non-specific queries will encourage respondents to reveal their true feelings. Skillfully interpreted - with attention to tone, body language and facial expression, as well as the actual words - the answers to these types of probes will give us the most useful direction.
7. Decompose copy into semantic vs. emotional components.
This applies to testing names and slogans in particular. Although consumers respond to stimuli holistically, their reactions in most cases are a joint function of semantic (verbiage) and emotional (executional) dimensions. Thus we test the name or line in plain type first - before showing it in finished executions. If a word has negative or confusing connotations, those attributes will not be totally neutralized by an execution. If we are testing “verizon,” we are going to learn more about its similarity to “horizon” or “veracity,” its sensory connotation (“eyes’) or maybe even feelings people have about the letters “v” and “z,” by testing it in plain type. This learning will open doors to additional execution possibilities. If we present the stylized name first, we are less likely to uncover these fundamentals.
8. Tolerate respondents who nit-pick.
We know the difference between concept and execution. Most (but not all) of our testing is against concepts. Focus group observers are frustrated by respondents who ask, “Why is she wearing glasses?” or simply say, “I don’t care for this typestyle.” But consumers see our materials holistically - no matter how many times we ask them to concentrate on the concept. If we scold them, we impair their identification with the task and make them think we only want to manipulate them. We must tolerate a certain amount of this. The risk of having respondents lose rapport and task identification is too great.
9. Use tangible individual props.
Creatives sometimes feel that the large concept board or package design that worked well in client presentations will be effective in focus groups. Not true. Whenever possible, give respondents their own personal copies of materials, even if it means a reduction in production values. More generally, physical props produce associations and deeper insights. A well-chosen map, a collage of competitive ads, a sampling of category products - just to look at - will stimulate productive discussion. When presenting materials for discussion, show, don’t tell.
10. Don’t ask unanswerable questions.
My favorite is: “Jim, just ask them what do we have to do to get their business.” This sounds like a simple request, one that cuts through layers of probing and inquiry and gets right to the heart of the issue - for the marketing manager, that is. The problem is it is not a question consumers can answer or care to answer. Too many assumptions are required. Plus, it’s offensive. It tells respondents we do not really care about them except as marketplace actors whose inner workings can be decoded and then exploited. And we actually ask them to help with the decoding! Knowing that the answer to a question will be helpful to decision makers is a necessary but not a sufficient factor in drafting the interview protocol. We must also know that a question is something respondents can address without offense to their dignity.
11. Anticipate “opinion leader” respondents in focus groups.
Although everyone in a focus group has the capacity for contribution, the discussions are often lopsided, with a small number of dominant participants. This is one of the main reasons for growth of individual depth interview studies over the past decade. If the potential for, and downsides of, opinion-leading are too high, one-on-one interviews may be a sound alternative. Another strategy in focus groups is having respondents answer key questions privately and in writing in advance of open discussion. This eliminates the potential for them to give conforming answers and it creates an opportunity for a “poll the group” procedure in which each person presents and explains his or her position. This lets commonalities of reactions surface objectively and it corrects any unbalanced “share of discussion” created by overly verbose panelists.
12. Extend the study to reach redundancy. (This applies primarily to focus groups.)
Research design balances what is needed for the decision vs. the investment in time and dollars, particularly the latter. Companies with larger research budgets do as many as eight to 10 focus groups in a single study. Others may go with only one or two. Upon finishing, say, two or three sessions, we know what we have but we struggle with whether it is valid or representative. In projects with more sessions we invariably reach a point in the second half when, upon concluding a session, we note: “We didn’t need that. We already knew what they were going to say!” This happens again in the remaining sessions and we go home confident in our conclusions. Big decisions have large downsides. Only by investing in a study of sufficient scope will one obtain the learning adequate for correct and confident decisions.