Advertising movies during the Super Bowl

All the right moves

Editor’s note: Don Bruzzone is president of Alameda, Calif.-based Bruzzone Research Company. Dan Rosen is senior vice president, Warner Bros. This article is adapted from a presentation delivered at the Advertising Research Foundation (ARF) Conference on Entertainment Research in Beverly Hills, Calif., on November 2, 2000.

Dan Rosen: I am here with Don Bruzzone, who is going to present the results of a study he fields every year, measuring the effectiveness of a very specific kind of TV advertising.

A few years ago, my management was wrestling with the idea of spending a lot of money to run a commercial on the Super Bowl. There was great debate within the studio management. Some felt that we absolutely had to be in the game — everyone watches it, and the commercials on the Super Bowl are always remembered and talked about for a long time after the game. Of course, filmmakers were especially excited about having spots for their picture on the game. But it costs millions of dollars to run a 30-second spot on the Super Bowl, and some members of upper management didn’t think it was necessarily the right thing to do.

So I looked around to see what opportunities there were to find out how well spots on the Super Bowl actually worked, and that’s how I found Don. He had been tracking response to Super Bowl advertising for about nine years, and had a body of data showing how well specific commercials performed. How well they were recognized, remembered, enjoyed, etc. So I did some business with him, and that’s what made him think I might be willing to join him in this presentation. I said okay, but I’m going to let him tell you how his research works and what we learned from it.

The wide differences in performance

Don Bruzzone: Figure 1 ranks the 32 movie commercials that have appeared on the Super Bowl since 1992 by recognition: the number that recognized them as commercials they had seen before. The darker bars show the other two measures used in evaluating the number that they reached and affected. The gray bars show the number that also remembered who they were for. The dark bars show the number that recognized them, knew who they were for, and liked them. Likability is the measure the ARF’s industry-wide validity study found to be the most predictive of a commercial’s actual effect on sales (Haley & Baldinger, 1991).

The first thing to note is that the percent that recognized them varied widely, from over 70 percent to under 10 percent. These results were for commercials that all had essentially the same broad initial exposure on the Super Bowl. Almost half of the population was watching under perfectly normal, real-world conditions. So it provided an unusual opportunity to find out what caused some to perform much better than others.

It is also worthy of note that by these measures, the Super Bowl movie commercial that reached and affected more people than any other in the past nine years was for the film A Few Good Men, the commercial that contained these memorable lines:

Tom Cruise: “All I want is the truth!”

Jack Nicholson: “You can’t handle the truth!”

It’s noteworthy because that may be a theme for this presentation. We have some findings that may not sit too well with the powers that be. There are two in particular that could provide an opportunity to see how well they can handle the truth. But, we are getting ahead of our story.

An average of 300 people across the country were surveyed about each commercial. It was done a couple of weeks after each game to see if the commercials had a lasting impact. A broad battery of diagnostic questions was asked. A wide variety of information about the respondents and the amount advertisers spent airing these commercials after the Super Bowl was obtained.

At the beginning we did this by mail using photo board questionnaires. In recent years we have found the job can be done better, faster and cheaper online. That incidentally is from the title of a recent article in Quirk’s that describes our Super Bowl methodology and the benefits we find in online surveys in greater detail (Bruzzone & Shellenberg, 2000). Mining this unique database showed a great deal about why some of these commercials reached and affected so many more than others.

The key segments of the market

First, we found two types of people were more likely to have noticed these commercials: younger respondents and those who go to the movies more frequently. Figure 2 shows the percent that recognized the average movie commercial declined dramatically by age, matching the pattern found in the percent that go to the movies regularly, which also drops rapidly with age.

People of all ages watch the Super Bowl and TV in general (see Figure 3). So, the evidence is quite conclusive. Younger respondents were more likely to notice these commercials. Older respondents were more likely to forget that they had ever looked at them. In effect, their mind ignored them.

The next pair of charts ranks the commercials in the same order as on the first chart. It shows that younger respondents were more likely to be reached and affected by all the movie commercials. And, with a few interesting exceptions, the commercials that reached and affected the greatest number tended to be the same for both old and young.

Differences are more dramatic when the results among regular moviegoers are compared to those who didn’t even say they went to the movies occasionally. Figure 6 shows a number of commercials performed much better among regular moviegoers, particularly when compared to the results in Figure 7 showing the number of respondents who indicated they didn’t go to the movies either regularly or occasionally. We will be exploring the reasons for this momentarily.

Segmentation strategy

Dan Rosen: Now we have to make some strategic decisions: Normally, we would concentrate our spending on regular moviegoers, because they buy most of the tickets. But sometimes we have a picture that we think will attract people who don’t go to the movies very often, thereby expanding our box office potential. One way is to split the buy and go after both audiences, which seems like the safest approach. I think Don has something to say about these alternatives.

Don Bruzzone: I’ve been known to refer to that last alternative Dan mentioned as the Great American Cop-Out: Do a little of each, not so much because it is right, but because it protects you in the boardroom from those who might climb up your backside and accuse you of picking the wrong strategy. What is the best strategy? We’ll leave the answer to that particular conundrum for another talk, and move on to the information you will need whatever strategy you adopt.

Expenditures

Respondents were surveyed two to four weeks after each game to see if the commercials had any lasting effects. During this period some of the Super Bowl commercials were aired again many times while others were never aired again. The next series of charts will show that differences in the amount spent to air the commercials during the month following the game accounted for about half of the differences in the number that noticed the commercials. It also provides some important evidence on the declining return you get when you keep a commercial on the air with a heavy schedule. It is the first of those potentially controversial findings alluded to earlier.

First, Figure 8 provides a picture of the amount spent to air each of these commercials on the Super Bowl and during the month after the game. The commercials are listed in the same order as on Figure 1, from best remembered to least remembered. It shows the differences in recognition are not explained by differences in expenditures. These expenditure figures are based on complete counts of all airings on networks during the period plus estimates of additional buys based on data from two spot markets. The basic expenditure data on which these estimates are based is from Nielsen’s Monitor Plus service, and they have all been adjusted to show the current value of a TV media dollar.

Figure 9 shows how efficiently the commercials reached and affected people. It takes the performance scores from the first chart, showing the percent reached and affected by each commercial and divides it by the amount spent, as shown in Figure 8. The results are shown in terms of the percentage points reached and affected per million dollars spent on either the Super Bowl or during the following month.

The number of commercials drops to 27 because in one of the earlier years, 1993, no one ordered analyses of expenditures and expenditure data was not obtained for that year’s Super Bowl commercials.

Return on investment

In terms of this ROI (return-on-investment) approach, we now have an entirely new set of commercials that proved to be best in delivering the most bang per buck. The differences that are accounted for by expenditures are shown on the next two scatter charts (Figures 10 and 11). They both show that differences in expenditures account for just about half of the differences in our measures of the number reached and affected.

When you divide recognition by expenditures you get a measure of how efficient a commercial is in capturing attention: the percent of the population that notices the commercial for each million dollars spent. In Figure 12 that cost efficiency measure is plotted against the total amount spent to air the commercial across the bottom. It shows a surprising fact. The only way to get a cost-efficient commercial is to limit expenditures to $2 million dollars. Then you can get as much as 25 percent to notice your commercial for each million dollars. But if you spend more than $2 million in this year’s dollars, you will have a hard time getting your efficiency up to a rate of even 8 percent per million.

These findings about Super Bowl commercials are not that unusual. They are in line with recent media research, particularly the research of John Philip Jones (Jones 1995). He conducted a major new study of the effect of advertising based on single-source data, where a household’s exposure to commercials was controlled through split cable techniques and their purchases were measured through coded scanner data. He found that the first time people noticed a commercial it caused a greater increase in their purchases of the advertised product than any subsequent exposure.

Previously the conventional wisdom was that a commercial had to be seen several times before it had its maximum impact on purchases (Naples 1979). Almost all the movie commercials in our study were being aired for the first time on the Super Bowl. What we found confirms that first airing was worth a great deal. Subsequent airings were worth much less.

You may have noticed that among the commercials we had expenditure data for, the largest amount of money - the equivalent of 20 million of today’s dollars - was spent to air the best-remembered commercial, for Dante’s Peak. That is five million a week, an impressive amount even if you are running for president.

So you may well ask, what do you mean heavy advertising doesn’t work? It produced a best-remembered commercial. So let’s be clear. This data doesn’t say spending a lot doesn’t work. It says it doesn’t work well. Figure 10 showed several commercials were able to reach half as many after spending only one-tenth as much.

That leads to a key question. How much should you spend?

Reach vs. efficiency

Dan Rosen: When we want to reach the widest possible audience and make the strongest impression, we run our campaign at the highest frequency level we can afford, especially in the week leading up to our opening. In some cases, we will also refresh the campaign, introducing new creative materials, often with reviews that come in close to our opening day. Don has learned some things about how well this can work.

Don Bruzzone: When you try switching to a new commercial, that hotshot in the boardroom may come after you: “Do you have any idea how much it costs to produce one of these commercials?” We don’t mean to over dramatize the situation, but this is Hollywood, so you might try a steely-eyed Jack Nicholson type rebuttal: “Do you realize how fast you can throw away millions airing worn-out commercials?” Leaving the dramatics aside, what we are showing is a process for coming up with the numbers that will quantify the debate, hopefully contributing to a more objective resolution of the issue.

What else accounts for the differences?

The amount spent to air these commercials only accounted for half the differences in the number they reached and affected. What accounted for the rest? We subjected the data to a variety of multivariate analyses to find out. Figure 13 shows the measures we used. It is the same battery of diagnostic measures our firm obtains for virtually all the advertising it tests. In this case it shows the diagnostic results for the best-remembered of all the Super Bowl movie commercials, the spot for A Few Good Men.

The boxes down the left of Figure 13 show words respondents can use to describe that commercial. These are combined into sub-clusters and major clusters as you move across the chart. Those on the top half reflect respondents’ emotional, affective, executionally-oriented reactions and end up in an overall measure of attitudes toward the commercial. Those on the bottom are the more thoughtful, rational, message-oriented reactions that funnel into an overall measure of attitudes toward the movie. Overall measures of likability, interest, awareness of the advertiser and the commercial’s effect on perceptions of the movie are shown on the right and along the bottom. When any score is in the most-favorable third it is called above average and it is shown in yellow. When it is in the middle third it is called average and is shaded blue. When it is in the least favorable third it is considered below average and is red.

This advertising response model (ARM) shows that reactions to A Few Good Men were predominately favorable. Almost all the measures are yellow. The only area where there are a number of below-average reactions is at the top in the entertainment cluster. Respondents who recognized this commercial were not as likely to consider it as amusing, clever, imaginative or original as the typical Super Bowl commercial. This ARM shows that the strengths that accounted for this commercial’s impressive success were in the areas of empathy and perceived relevance. It shows you don’t have to be funny to be successful on the Super Bowl. We should also note in passing that the one area where this commercial exceeded our norms by the greatest amount was in clarity. Hardly anyone came away from that commercial feeling any confusion over what that movie was about.

Figure 14 is an example of an ARM from the other end of the success spectrum, the animated science fiction film Titan AE.

On this next ARM, we find hardly any performance scores that are above average and a lot that are below average. Those are concentrated at the bottom. The overall measures of likability, interest and the rest are all below average, confirming what we have already seen in the rankings. The primary driving forces are at the lower left. Those who noticed this commercial didn’t find it effective, believable, or worth remembering. The characteristic that got the lowest rating of all was clarity. Respondents found it pointless and extremely confusing.

We used the same approach to analyze differences between population groups. We will use the Austin Powers commercial as an example.

The next ARM (Figure 15) shows how Austin Powers performed among those under 30. Its performance was above-average in almost all areas. It lacked a bit in warmth, they didn’t find it very credible, and even among those under 30 there was a greater than usual number that called it silly, phony and irritating. But those reactions were more than offset by the number that saw it as entertaining, appealing and something they personally found very relevant.

Contrast that with the reactions to the same Austin Powers commercial by those 30 and over. It doesn’t look like they are reacting to the same commercial, but they are. This older group found its humor and entertainment value average, but not exceptional. Where they really got turned off was in not seeing any warmth or appeal in the Austin Powers character, or in seeing anything in the commercial they found relevant to themselves. It struck them as a commercial for someone else.

As one final example, take the differences between regular moviegoers and those who seldom or never go. The Super Bowl commercial for Lost in Space was one where reactions differed dramatically between the two groups and accounted for the wide difference in performance noted earlier.

First, reactions from regular moviegoers are shown on the ARM in Figure 17. All major measures are favorable. Here the most exceptional ones are the number that found it appealing, fast-moving and lively. Further, there was not a single regular moviegoer who found this commercial confusing.

Compare that with reactions from those who noticed the commercial but seldom or never go to the movies. That is shown in Figure 18. They didn’t find it entertaining. It didn’t look like anything they were familiar with. That pulled the total for attitudes toward the ad down to a below-average level. They didn’t find it convincing, effective or credible. And the number that called it confusing was the highest we ever found for a Super Bowl movie commercial.

Analysis

What we have been looking at is the type of data we evaluated for every Super Bowl movie commercial. We used three types of analysis to identify the characteristics most frequently associated with success, or the lack of it: regression, discriminant analysis and CHAID. We found they were all in general agreement about the key factors driving success.

Humor
The first is one you might expect. The Super Bowl movie commercials that people were most likely to notice, and least likely to ignore, tended to be the commercials that were the funniest and the most entertaining. In our respondents’ opinions they were the commercials for Nutty Professor II, The Wedding Singer, Austin Powers, Groundhog Day, Liar, Liar, Stop! Or My Mom Will Shoot, Wild Wild West.

We have already mentioned well-remembered commercials that were exceptions, Dante’s Peak and A Few Good Men. Their success was more than offset by the success of the humorous commercials shown above. But right off the bat we have started showing exceptions to the findings, something that seems to be par for any serious attempt to establish guidelines for successful advertising.

Clarity
The second major finding may come as more of a surprise: Don’t confuse people.

Respondents found lots of movie commercials on the Super Bowl very confusing. This is the second of the potentially controversial findings we alluded to earlier. A study of the commercials respondents found confusing suggests two basic problems. Explosions and special effects may be the pride of the producer, but pack them all together in a 30-second commercial, without any background on what is happening or why, and you can have a commercial that a lot of people can find confusing, meaningless and easy to ignore. The second underlying problem is similar. When you have a whole movie to select shots from there seems to be a tendency to select too many. It’s as if those involved with the movie have grown quite attached to what they see as the great scenes in the movie, and they try to get at least a few frames of every one of them in the commercial. And, again the effect is more confusing than impressive.

The solution to confusion is to make a commercial into something that people can make sense of, something that is understandable and has some meaning. Meaningfulness can take many forms. It can be a joke. When you don’t get the joke, you are likely to call the commercial confusing. When you don’t understand what all the action is about, you are likely to call it confusing, not fast-moving and lively.

This ties into the finding 10 years ago that likability is the best predictor of advertising’s effect on sales. We were proud to play a role in the watershed research that showed the most important characteristic of likable advertising is meaningfulness (Biel, 1990). People don’t like things they don’t understand. One creative strategy that seldom worked on the Super Bowl is the suspense format, where you try to create curiosity about the unknown. It is part of the same principle. For most people it is almost unconscious. They don’t have the patience for confusion in advertising. If they don’t grasp the situation in the first few seconds they lose interest and start ignoring it. When you go back later you find no traces of that commercial in their memory, and no effect on their behavior. It is a lesson we found is often ignored in advertising movies on the Super Bowl.

On the other hand, when this lesson about clarity, meaningfulness and likability is followed, you can get exceptional results. Consider the two exceptions we have been talking about. There was nothing funny about A Few Good Men, but its content was crystal clear. It was a movie about a court-martial. Dante’s Peak certainly had explosions and special effects. But you heard Pierce Brosnan say the mountain is going to blow, it did, and you saw him racing to get away. In 30 seconds you had the complete storyline for the movie.

We have already seen that some commercials did an impressive job reaching and affecting large numbers of people after much less exposure than most of the commercials got. Those were the commercials where the basic theme was clear. You didn’t have to see it a second time to figure out what it was all about. Clarity is always something to strive for.

Movie stars
The third factor that contributes to success in advertising movies on the Super Bowl is certainly no surprise to Hollywood: a big name star. When people see a face they know and like they stop and pay attention. You don’t have a big name star in your movie? You have to make up for that with strengths in other areas. The drivers of success are almost all interchangeable.

Empathy
Empathy provided an alternative way of capturing attention and interest. It did not account for as many of the Super Bowl successes as humor and entertainment value, but it accounted for enough to qualify as one of the major driving forces. Shining Through, a 1992 movie with Michael Douglas and Melanie Griffith escaping the ravages of war, reached and affected a lot of people because of the number that found it appealing and well done. Clint Eastwood’s 1999 True Crime, about a reporter trying to save a man from execution, got the highest scores for warmth and sensitivity.

Those who feel humor and slapstick provide the only route to success on the Super Bowl have not studied the results.

Reaching and affecting the sub groups

For regular moviegoers the most important factor was “freshness.” Expenditures had less effect on this group. They noticed movie commercials even when they had very little exposure. The commercial for Mercury Rising is an example of a commercial that ranked much higher among regular moviegoers, where it was fourth. It dropped to 29th among those who seldom go to the movies.

Those who don’t go regularly showed somewhat the opposite tendency. They were more likely to be reached and affected by movie commercials that dealt with familiar themes. The commercial for Falling Down performed well among those who don’t go to movies regularly, and whose theme they considered familiar.

Young folks were reached best by commercials that were entertaining. They were drawn to commercials they called silly. That included almost all the commercials we already showed as humorous commercials. But the commercial for The Getaway is another example of a top performing commercial that performed exceptionally well because younger respondents found its excitement very entertaining.

Older respondents were the ones for whom the amount of exposure made the biggest difference. They favored commercials they considered clever and well done. The commercial for the WWII action film U571 is an example of a commercial that older respondents felt was unusually well done.

Summary

Dan Rosen: So, the net of all this seems to be:

• First - not a big surprise - entertainment values, and especially humor, are the most significant elements in effective movie advertising.
• Next - and again, no surprise, but often not heeded - clarity, ease of understanding, directness of the message, don’t confuse viewers...lots of Super Bowl advertising didn’t work because viewers didn’t understand the message.
• Of course, movie stars are the main reason why people go to the movies, and they help enormously in advertising - they get people’s attention, and they make the ads memorable.
• Emotional content is also an effective element when incorporated into the advertising in the right way and not overdone...we can manipulate the viewer, but we shouldn’t be too obvious about it.
• Be fresh and innovative. Don’t do the same thing all the time. People get bored and tired and tune out.
• To reach the infrequent moviegoer, who tends to be older, be conventional. This part of the audience won’t stay with us if they don’t recognize what we are showing them. And, show it to them over and over again until it sinks in. On the other hand, frequent moviegoers, especially the young ones, respond best to humor and even silliness.

Of course, advertising is only one part of the mix. Publicity, word-of-mouth, reviews, competition...these are all important elements in creating box-office.

But the one element we can control and measure is advertising, and I hope we have shown you some of the ways advertising works today.

References

Alexander L. Biel and Carol A. Bridgwater (1990), Attributes of Likable Television Commercials, Journal of Advertising Research, June/July 1990.

Donald E. Bruzzone and R. Paul Shellenberg (2000), Track the effect of advertising better, faster, and cheaper on-line, Quirk’s Marketing Research Review, July/August 2000.

Russell I. Haley and Allan L. Baldinger (1991), The ARF Copy Research Validity Project, Journal of Advertising Research, April/May 1991.

John Philip Jones (1995), Single Source Research Begins to Fulfill its Promise, Journal of Advertising Research, May/June 1995.

Michael J. Naples (1979), Effective Frequency: The Relationship Between Frequency and Advertising Effectiveness, New York: Association of National Advertisers, 1979.