Keep the lawyers at bay

Editor's note: Jerry Thomas is president and chief executive of Decision Analyst Inc., an Arlington, Texas, research firm. 

Senior executives are competitive types, typically, and want to confront the competition head-on. Their mantra seems to be “Let’s drop a bomb down the market leader’s smokestack” by a frontal attack with comparative advertising claims. “Let’s advertise that our product is preferred by consumers two-to-one over the market leader’s brand.”

This type of bravado appeals especially to senior marketing executives, who see it as a great way to gain market share at the expense of a hated rival. The senior vice president (SVP) of marketing calls in his consumer insights team and says, “We want hard evidence, proof, that our product is preferred over the market leader’s product two-to-one. This is a rush project, of course, so get to work.”

The first question to ponder: Is it a good idea to make a head-to-head comparative claim against a competitor? First, consumers tend to think that brand-to-brand comparisons are in poor taste, unfair and unsportsmanlike, so it’s possible the comparative claim could backfire. Second, consumers are often skeptical of comparative claims in advertising and tend to discount them. Third, it’s possible that your great head-to-head comparison claim will be remembered as advertising for the better-known competitive brand. Fourth, the competitive brand’s management team might take offense at your advertising claim and file a nasty lawsuit.

So, before your marketing executives rush off to create that great head-to-head commercial, a recommended best practice is to test a number of different advertising claims or messages to see which types of claims resonate with your target audience. Chances are you will find claims better than the brand-versus-brand comparisons. However, there’s usually no time for such claims-screening, because in today’s fast-paced, throw-it-against-the-wall, skip-the-research, agile world, the SVP of marketing is jumping up and down and hell-bent on directly attacking his nemesis, the market-leading brand. What the heck do you do?

Very often, the researcher might be asked to find evidence in existing studies to support a new head-to-head advertising claim, or asked to go out and test lots of comparative claims in one big study. These are fools’ errands. Your competitor’s lawyers will subpoena all of your brand’s research studies and expose them in open court and then take glee in every little methodological flaw in every study. (And, by the way, each and every one of those studies in your files can be picked to pieces by a good lawyer. Your studies are not nearly as perfect as you believe.) The plaintiff’s lawyers will have a jolly good time revealing all the evidence in your research studies that disprove or countermand the exact claim your SVP wants to make. How can you develop evidence to support your advertising claim, that will be accepted by the major networks and hold up in court should your company become the defendant in a lawsuit?

No generalities

Let’s suppose you did have time to do a claims-screening survey and you did find a comparative claim that appeals to your target audience. To construct a defensible research design, you must know exactly what comparative claim the SVP of marketing wants to make. The claim tested must be the same, exact language that will be used in the advertising. No generalities, no broad strokes. Precise language is a must.

For example, the claim might be, “A majority of U.S. adults prefer the taste of Heinz Ketchup over Hunt’s Ketchup.” This is an assertion you can test; you can design research to prove or disprove this claim. Another example, “The taste of Skippy Peanut Butter is preferred two-to-one over the taste of Peter Pan Peanut Butter by U.S. children.” Apart from ambiguity over what “two-to-one” means, this is a specific claim that probably can be tested.

Oppositely, you must avoid trying to design research to support general claims, such as “Skippy Peanut Butter is better than Peter Pan Peanut Butter.” What exactly does “better than” mean: its color, its packaging, its ingredients, its texture, its price or what?

The claim determines the design

Once you have a highly precise, specific comparative claim to test, then you can design research to prove (or disprove) that specific claim. The specific claim largely determines the research design. Qualitative research (focus groups, depth interviews, online forums, etc.) generally cannot be used to substantiate advertising claims. Sample sizes tend to be small, the qualitative discussions will contain countervailing evidence and the open-ended natures of the answers do not lend themselves to statistical significance testing. Qualitative research is not an option.

If you can prove your claim with credible secondary data, that might be a workable solution, unless that secondary data source also contains information in conflict with your claim. Usually, however, you will have to do primary research in the form of a survey to precisely determine if the proposed comparative claim is true or false. Some rules of thumb:

• A large sample is better than a small one. The sample must be large enough to be statistically significant at a 95-percent confidence level – or 99 percent, which is even better. More important, the sample must sound “large” to the judge, jury or review board, regardless of statistical significance. A sample size of at least 1,000 target-audience respondents is typically recommended.
• The sample must represent the whole market, as defined by the specific claim. So, in the previous example, “A majority of U.S. adults prefer the taste of Heinz Ketchup over Hunt’s Ketchup,” the sample must consist of “all adults” in the U.S. (that means all 50 states, plus Washington, D.C., plus U.S. territories). Adulthood is generally defined as beginning at age 18, so the sample would be anyone 18 years of age or older living in the defined geographic areas.

You will note that the sample would include U.S. adults who do not use ketchup, because your claim did not place any limits on “U.S. adults.” The sample must be representative of all U.S. adults, including ethnic minorities, men, women, high income and low income and all geographic areas. A random sample, or as close to random as possible, is the most defensible sample. A stratified random sample can be defended as an improvement upon a purely random sample.

However, if you should choose to do 100 surveys in each of the 10 most populous metro areas in the U.S., that would not be an acceptable sample. The adults in the top metro area (i.e., the metro area with the largest population of adults) would be less represented than the adults in the 10th largest metro area; and all the other metro areas and rural areas would be omitted from the sample. Remember, the sample must represent all U.S. adults equally.

• When designing the questionnaire, other than some demographic questions to prove the sample is representative, you should ask only one question. If you ask a number of questions about the two ketchups’ taste, for example, you are only creating a playground for opposing lawyers to discredit you in front of judge and jury.

Different questions will create different results and the lawyer will use these differences to weaken your claim and destroy your case. To continue with the ketchup example, the one question might be worded as, “Which one of these ketchups do you prefer, Heinz Ketchup or Hunt’s Ketchup?” To eliminate positional bias, you would, of course, rotate or randomize the order of Heinz Ketchup and Hunt’s Ketchup in the question. But, wait a minute, something is wrong with our question. The advertising claim we hope to prove (“A majority of U.S. adults prefer the taste of Heinz Ketchup over Hunt’s Ketchup”) will not be proven by the proposed question, because we did not include the key word “taste” in our question.

OK, so let’s revise the question to read: “Which one of these ketchups tastes better to you, Heinz Ketchup or Hunt’s Ketchup?” Again, we would randomize the order of the brand names in the question. We are getting closer to an ideal question but we are not quite there. The wording “…tastes better…” is not exactly the same as “…prefer the taste of….” Remember, our question wording should ideally use the identical language as the advertising claim. So, we finally decide on the exact question: “Which one of these ketchups do you prefer the taste of, Heinz Ketchup or Hunt’s Ketchup?”

Darn it. We are closer still but the question is not yet perfect. So, we think some more and craft this final question: “Do you, personally, prefer the taste of Heinz Ketchup over Hunt’s Ketchup; or do you, personally, prefer the taste of Hunt’s Ketchup over Heinz Ketchup? The order of the two sentences in the question would be randomized, of course. Our answer choices would be:

Personally prefer the taste of Heinz Ketchup over Hunt’s Ketchup

Personally prefer the taste of Hunt’s Ketchup over Heinz Ketchup

The order of these two answer choices would be randomized, of course. Oh, but we forgot something. What if someone did not prefer one ketchup over the other? We must add in a “no preference” answer choice, such as “Do not prefer the taste of one ketchup over the other.” Now, the questionnaire is ready.

• To achieve a representative sample in this example, the recommended method would be to recruit a national probability sample of U.S. adults who agree to participate in a ketchup taste test and then ship a bottle of Heinz Ketchup (branded), along with a bottle of Hunt’s Ketchup (branded), to each respondent.

Participants would be asked to use both ketchups in randomized order over a period of several days and then complete an online survey about their experiences. During the follow-up survey, your carefully crafted taste question would be asked. You could do a blind taste test (i.e., unbranded) rather than branded but then you would have to change the claim to fit the blind methodology.

As you go about designing the research, creating the sampling plan, crafting the questionnaire and anticipating the analysis, you normally leave a paper trail (and an electronic trail). For any research that might end up as evidence in a lawsuit, you do not want a paper or electronic trail haunting you. Such records are a boon for opposing lawyers. A best practice is to rely on phone conversations and in-person discussions (without any audio recording devices present or written notes lying around). Then, once everyone is in agreement, you put down on paper (or in computer memory) the one plan, or one questionnaire, that sums up your decisions. If you leave a paper trail, you might have to explain to the judge and jury each of the 10 drafts of the final questionnaire and why you changed each and every word from one draft to the next. Leave as few tracks as possible. Of course, you will need documentation of exactly what you did, and how you did it.

Attacked by the lawyers

Regardless of the research design and the data collection method chosen, your survey will be successfully attacked by the opposing lawyers. All of our survey research methods have flaws that smart lawyers will exploit in court. In the example of the Heinz-Hunt’s in-home usage taste test, for instance, the opposing lawyers would point out that a) you did not personally visit any of the test households and observe, and can’t prove who actually took the survey; and b) nor can you prove the participants actually tasted the two ketchups; and c) nor can you prove participants did not confuse the two brands and answer incorrectly, etc., etc. Every other data collection method can also be ripped to pieces by well-prepared opposing lawyers.

Most courts, however, will accept survey-based evidence, if the research company’s reputation is solid, if the research firm is independent and objective and if it appears that the research methods are reasonably sound. You must use a professional research firm for claims substantiation research; it’s not the time or place to try out your do-it-yourself survey skills.

Years ago, a consultant to one party in a legal dispute (a client of ours) decided to save some money and do his own survey and present the results in court. The plaintiff’s lawyer cross-examined him without mercy until the poor fellow ended up vomiting in a nearby trash can. Cross-examinations are somewhat akin to having splinters shoved under your fingernails.

Speak softly

While there is much more that could be said about advertising claims research, the key point is the importance of seeking professional legal help and professional research assistance. Few advertising claim disputes ever actually go to trial but you can spend millions of dollars and waste months of time producing subpoenaed documents, answering interrogatories, preparing for depositions and suffering through depositions.

Some advertising claim disputes are resolved by media network feedback, some by cold feet, some by bankruptcy and some by wisdom. Excessive bravado puts a company and its brand at the greatest risk of a court appearance. So, speak softly and leave that “big stick” at home.