Editor’s note: Robert J. Morais is a principal at Weinman Schnee Morais Inc., a New York research firm.
Advertising agency executives have long been wary of recall testing. They worry that single-exposure recall tests do not represent the impact of multiple advertising exposures and that guidelines for generating high recall inhibit creativity. They contend that advertising recall is less important than persuasion. They fret that recall scores are lower for new products and services. Some agency executives complain that low recall scores lead to uncompensated agency labor on new, clutter-busting campaigns that clients demand. In addition to these concerns, numerous agency executives and many clients believe that recall scores have edged downward recently. Executives attribute this movement to the distractions of the Internet and video games, increasing advertising clutter and demands on consumers’ attention in the first decade of the 21st century.
If advertising recall has declined over the years, comparisons with historical databases must be adjusted or databases must have a limited time frame with norms that reflect only the past five to 10 years of data. If recall has remained constant, such adjustments, and the attendant worries, are unwarranted. If recall has increased over the years, the industry will need to adjust expectations upward.
We decided to examine the database of television commercials that we recall-tested over the past few decades. The segment of commercials we analyzed consists of over-the-counter health and beauty aids and household products. These categories limit the scope of our report; we do not claim that our findings represent recall scores for automobiles, financial services, fashion, telecommunications, soft drinks and countless other types of advertising. We do feel that the findings merit consideration by all marketers and their advertising agencies. We invite executives engaged in categories that we have not analyzed to conduct a longitudinal study of recall scores and share them with our industry.
For our analysis, we mined our database of over 500 30- and 15-second television commercials. Our recall test method entails recruitment of a random sample of respondents to watch an on-air TV show who are told that they will be called back the following day for their reactions to the program. The day after exposure of the advertising, respondents are asked if they saw a commercial in the category of the test commercial. If a further recall aid is needed, they are given a cue for the product’s brand name. We look at two key recall measures: related recall, when the respondent is able only to play back a generic sales message that is universal to the category, and proven recall, which is met when the respondent can describe specific visual or audio elements in the commercial or express a unique sales message from the commercial. We divided the 30-second commercial database into two segments, 1978-1991 and 1992-2005, and the 15-second commercial database into two segments, 1985-1995 and 1996-2006. We compared mean recall results for each commercial length across the time periods.
For 30-second commercials, we found that average related recall dropped from 24.7 percent to 22.5 percent and that proven recall was flat, from 21.7 to 21.5. For 15-second commercials, related recall decreased from 18.7 percent to 16.6 percent and proven recall from 17.4 percent to 15.6 percent. We isolated more recent data and found a slight uptick for related and proven recall, but the larger, longer-term data sets are more indicative of a trend. We suspect that the long-term changes are linked to increased commercial clutter and distractions like the Internet. Our methodology controls for some distractions because we instruct respondents to tune into a particular program, but we can not control whether or not they are surfing the Internet while watching the program.
What can we conclude from these findings? Advertising executives and marketers can relax a bit; recall scores in the categories we tested have not tumbled dramatically. Still, a two-point decrease in advertising recall is statistically significant for our database. Executives who correlate that reduction with lost sales will certainly worry - and they will press harder to create advertising that is remembered. For many companies, this change is so small that they will not see cause for concern. Companies that attribute the downward recall trend to competition from other media may become more aggressive in adopting online, word-of-mouth and other less traditional venues.
Don’t overreact
We suggest that recall tests continue to be read closely but we do not recommend overreacting to the trend that we uncovered. Advertising research is an aid to, not a substitute for, judgment and recall testing is one of several tools that support our judgment. Assessments of comprehension, likeability, persuasion and other variables also play a key role in the decision-making process. Professor Joel S. Dubow suggests that we think of “recall as the ignition system of an automobile and persuasion as the fuel system” and notes that “the fuel system doesn’t get a chance to perform its function if the ignition system is too weak to start the car.”
Now that we know that the ignition system is not performing the way it used to, we should check the fuel gauge. Have norms for persuasion and other communication measures also changed in the past 20 to 30 years? That question should be examined on another day.