Editor’s note: Charles Young is CEO of Ameritest, an Albuquerque, N.M., research firm. Stephen Sands is founder and chief science officer of Sands Research, El Paso, Texas.

Currently there is a great deal of interest among advertisers in new biometric research techniques that have been developed to deepen our understanding of how TV commercials really work. Physiological measures of various kinds - including electroencephalography (EEG) measures of brain waves, galvanic skin response, heart rates, facial response, pupil dilation and new brain imaging techniques such as functional magnetic resonance imaging or fMRI - are being used in an attempt to explore the underlying neurological basis of advertising effectiveness. These approaches are appealing because of their promise of providing grounding for the “soft” social science of advertising research in the “hard” neuroscience now being done on the brain.

But the reaction of many brand managers and creative directors is like that of family visitors to a hospital room nervously watching the readouts on the monitors hooked up to their television commercial. “What does it all mean?” they ask.

Driving ad researchers’ interest in these new diagnostic methods is the sense that traditional interview-based methods of studying commercials fall short. One reason for this is the feeling that “self-report” data about how ads work is inherently suspect because the consumer may not know his or her real mind. Brain processes that are operating below the level of the conscious mind - the subconscious - cannot be self-reported.

Another reason for interest in biometrics is that traditional methods using rating statements and open-ended questions are verbally-biased and do not give proper credit to the non-verbal component of commercials, such as music or the visuals. Creative people have long suspected that traditional research approaches, such as day-after-recall testing, unfairly reward overly rational advertising and penalize emotional executions.

In fact, traditional research constructs such as attention-getting power, recall, brand linkage, communication and motivation measures (e.g., purchase intent) have been the subject of a great deal of validation research over many years and most of them work pretty well in predicting in-market ad performance. Report-card measures like these will always be relevant to practical advertising professionals who have to make go/no-go decisions about their advertising investment. The real contribution of biometrics and non-verbal measurement is more likely to come in the form of improved diagnostics, providing new insights into why, for example, some ads get our attention while others do not.

Building bridges

Our own view is that useful knowledge in advertising science will come about as a result of building bridges between established ad research methods and the new findings of neuroscience. Indeed, neuroscientists have a unique opportunity that other medical researchers do not. This is because the human brain is the only organ in the body that can accurately self-report anything at all about what is happening to itself. What secrets would cardiologists learn about heart attacks if hearts could talk?

In the spirit of cooperating to gain new knowledge, Ameritest, an Albuquerque, N.M., ad research company, teamed up to do an R&D experiment with Sands Research, a brain-scan company based in El Paso, Texas. Our goal was to learn from each other by comparing research results on some ads from our two points of view: one, a measurement system based on Internet self-report, picture-sorting data and the other based on EEG brain wave measurement of ad effectiveness.

For our experiment we looked at 36 television commercials which Ameritest researched in the past year for fast-food advertisers, representing a range of well-known brands, from McDonald’s to Pizza Hut to Subway. These ads were tested online, as part of Ameritest’s syndicated creative tracking service, with a national sample of nearly 3,000 respondents. Sands retested these same ads among 60 respondents at a fixed location with its EEG equipment.

In our experimental design we wanted to understand the differences in performance between two groups of commercials in our sample. In the Ameritest system, 20 of the ads had performed well, scoring statistically above-norm in attention-getting power, but 16 ads had performed poorly, scoring below-norm. Attention is an important primary measure of ad performance because the first job an ad has to do in our cluttered world of advertising is to get noticed.

The Ameritest measure of attention is a verbal response to a simple question that we ask after a respondent is shown a clutter reel of competing ads: “Which of these ads did you find interesting?” This overall measure of advertising interest has been validated for predicting in-market ad performance, not only for awareness levels but also, in the case of fast-food advertising, for predicting sales results.

Contradict each other?

Using attention as a starting point we wanted to find out what each of the moment-by-moment diagnostics of our respective research systems would tell us about the breakthrough power of these ads. Will they tell us the same thing? Will they contradict each other? Or will they complement each other? While both are non-verbal techniques, the two diagnostic approaches are decidedly different.

With an EEG the electrical activity of the brain is measured in real time, to the millisecond, to gauge how “aroused” the brain is by the inputs it’s receiving from some stimulus. The EEG machine produces a very fine-grain curve showing viewer response; spikes show multiple peaks of arousal at different moments as a respondent watches a 30-second commercial.

Moreover, with Sands’s measurement cap, containing 68 strategically-placed electrodes, feeding data through a proprietary, noise-reducing software, it’s possible to measure how much electrical energy is being generated and determine where on the outer surfaces of the brain the activity is coming from. This shows the researcher which different parts of the brain are aroused at different moments in a commercial.

In contrast, Ameritest Picture Sorts use photographic stills taken from the ad to probe respondent reactions about 20 minutes after initial viewing. Multiple sorts are used to probe respondent reactions to commercials on multiple levels of self-reported response - the Flow of Attention produces different insights than the Flow of Emotion or the Flow of Meaning - and companion Copy Sorts are used to deconstruct the role of the copy from the role of the visuals in the overall performance of a commercial.

The sort we are looking at here is the first sort, the Flow of Attention, which asks respondents to sort images simply based on those they remember seeing in the ad versus those they don’t. The resulting graph of selective attention is, like a brain wave curve, very fine-grained and bumpy - a typical commercial produces multiple peak moments of audience attention during a 30-second spot.

More peak moments

So, what did we learn from our experiment? When we counted the number of peaks produced by the brain wave machine and the number of peaks produced by the Flow of Attention and correlated them with the attention scores for the fast-food ads in our sample we found that both diagnostic techniques are predictive of commercial breakthrough performance. As you can see in Figure 1, high-attention ads generate significantly more peak moments of attention during the flow of the commercial than low-attention ads do - a statement which is true regardless of whether you are talking Flow of Attention peaks or brain wave peaks.

The second thing we learned is that the peak moments identified by the two diagnostics are not always the same moments. If you look at Figure 2 you can see the overlap between the two. For these 36 commercials, brain wave measurement identified 113 peaks of arousal and Picture Sorts identified 149, but only 61 of these moments, or roughly half, were the same moments. What this tells us is that each approach has something to teach the other about how the brain functions.

We get new insights into how advertising works by looking at the actual content of the ads at these different points in time. Let’s start by looking at the moments where the two systems agree.

Double peaks - that is, moments that are peaks on both the brain wave and the Picture Sort graphs - are by far the most strongly predictive of the overall attention score for the ad. The convergence of the two systems, therefore, helps us identify the most hardworking parts of a television commercial. Analysis revealed that double peaks were most likely to contain the following content:

  • important news, such as the announcement of a strong price promotion;
  • inciting incidents, typically involving a moment strongly charged with negative emotion to set up a joke or storyline;
  • surprising moments or turning points in stories;
  • climactic moments or punch lines.

In other words, double peaks can be used to identify the dramatic highlights of the ad from the audience’s point of view. If we were talking movies instead of television commercials, these would be the scenes that would make it into the movie trailer.

Now, what of the peak moments that one diagnostic technique identifies but the other does not? One part of the answer is quite simple. Remember that with brain waves we are measuring the audience response to the total multichannel experience of the commercial - the pictures plus the words plus the music - while the Picture Sorts focus on the vision part of the television commercial. In many instances, brain waves peak at a moment in the ad identified with a strong line of copy - an effect we can see in our Copy Sorts but not in our Picture Sorts.

Intriguing finding

But that’s not the whole answer. A more intriguing finding is one that helps explain the rhythmic structure of visual communication that we’ve seen over the years in the wave-like patterns produced by the Flow of Attention graphs.

As we analyzed these commercials scene by scene, we observed the phenomenon that the brain wave pattern would sometimes peak at the beginning of a scene, usually where it was unclear what was going on, which would be a low spot in the Flow of Attention, and the brain wave arousal would fade at the end of the scene, just as the Flow of Attention was peaking. In short, the brain wave curve and Flow of Attention curve would be moving in opposite directions (see Figure 3).

This makes sense if we consider the differences between our two measurement techniques. Brain waves measure the amount of energy being produced by the brain moment-by-moment in real time, as a consumer watches an ad. The first Picture Sort is called the Flow of Attention because it is a measure of selective attention and is, in fact, most predictive of the attention score of an ad. But because visual recognition is collected after the fact, some minutes after images have been sorted out by the brain and encoded into long-term memory, the Flow of Attention graph can also be interpreted as a map of remembering and forgetting. The higher a picture is plotted on a Flow of Attention graph the more a consumer audience remembers that image.

Memory researchers have established that what the brain remembers is the meaning of what happens, not simple sensory inputs. With that in mind, the difference between what brain waves are peaking on and what the Picture Sorts are peaking on gives us a new insight into how long-term brand memories are created by advertising.

This is how the process works from a storytelling perspective: Scenes in stories open by arousing curiosity - engaging the audience with a change in the direction of the film. In response, the brain releases electrical charges to alert the mind to the new situation. “Pay attention, something new is about to happen here!” Brain waves peak as neurons fire away. As the scene unfolds and then resolves, the important information in the scene is sorted out. Finally the brain can return to a state of rest and neuronal activity settles down.

As a result, the mind may only store the end of the scene - when the meaning of the scene is resolved - as a memory. This is the part of the scene that would generate a peak in the Flow of Attention. In the context of advertising, it would be these meaningful memories that comprise the set of visuals encoded into long-term memory that we would expect to be most strongly associated with the brand image.

Of course, brain waves may also peak on a resolution scene. Because a commercial is a chain of meaningful moments, the resolution of one scene may be the beginning of another as involvement is ratcheted up with the increase in audience engagement in the flow of the film from the beginning to the end of a strong commercial.

Much we don’t understand

An interesting aspect of Sands’s approach to measuring brain activity is to see how different parts of the brain light up as different meanings are created by parts of the ad. This reminds us all that there is much that we don’t understand about the complex brain processes, many of them subconscious, which operate as a consumer watches a television commercial, a Web video or a movie.

With brain waves we can know precisely when the brain gets excited and approximately where in the gray matter the electrical activity is located - but we don’t know what thoughts and what kinds of feelings the subject was having at any given moment. For that we need a mind-reading machine. Fortunately, the introspective mind can read itself - up to a point.

We could, of course, simply ask the consumer what he or she was feeling and thinking while watching the ad. By aiding the questioning with still photos from the ad associated with the precise moment when brain activity peaks, we aim to help the brain reconstruct the scene of the experience step-by-step.

With the second Picture Sort, the Flow of Emotion, we get at how much emotion, either positive or negative, the consumer was feeling at any given point in the film. And with the third Picture Sort, the Flow of Meaning, we get at which kinds of thoughts or emotions the consumer associated with that moment. In the context of these fast-food commercials we gave the consumer 10 brand values to choose from, to tag the meaning of each image.

Several images from a high-scoring Wendy’s commercial, shown in Figure 4, illustrate this. In this commercial a man tells his girlfriend a story about why he was late coming home: shortly after stopping at Wendy’s for a 99¢ Crispy Chicken Sandwich, he encountered a real unicorn in the street. As we see, in flashback, the remarkable sight of a white unicorn standing in the middle of the street the girl interrupts, unbelieving, “Wait a minute! There’s a 99¢ Crispy Chicken Sandwich at Wendy’s?!”

Three images, identified by peaks on both brain waves and Picture Sorts, convey the gist of the story. In the first, we see the man eating the chicken sandwich. Second, we see the unicorn. Finally, we see the unbelievable price offer.

The Flow of Meaning allows us to reconstruct the sequence of brand-building ideas consumer took away from this commercial: 1) Good-tasting food, 2) at an enjoyable place to eat, 3) at a great price. (Admittedly, the rating of enjoyable place is low, but it makes an important point - advertising imagery can work on a metaphorical and not just a literal level. Associating the enchanting images of unicorns can subliminally imply that Wendy’s is an enjoyable place and consumer self-report data can provide important insights to how such imagery can impact brand perceptions.)

Importantly, notice that for each of the three meaningful images different parts of the brain light up! This is strongly suggestive that different types of visual content in an ad are processed in different parts of the brain and, ultimately, may be stored in different memory systems of the mind.

Important for branding

Our hypothesis is that three different memory types are important for branding. First, there are knowledge memories, which consist of semantic ideas and rational concepts (e.g., the 99¢ price offer). Second, there are emotion, or episodic, memories, which are made up of the events, characters and relationships we describe in stories (e.g., the unicorn sighting). Third, there are action, or procedural, memories of bodily experiences and physical sensations (e.g., the sensory act of eating the chicken sandwich).

Images of action, such as product-in-use or bite-and-smile shots, and images that evoke the senses of touch, taste and smell occur frequently in television advertising. But the role such images play in an ad is usually overlooked in the perennial debate of rational versus emotional advertising. The visual evidence provided by Sands’s brain wave data strongly suggests that they provide a distinct class of brand imagery. Such imagery may be important for providing the consumer with the opportunity to rehearse consuming the product in the mind, as a form of virtual consumption before he or she consumes the actual product in the real world.

A final interesting observation is that we found more Picture Sorts peaks than brain wave peaks in these fast-food commercials. It turns out that there are nearly the same number of Flow of Attention and brain wave peaks in the story part of an ad - but the Flow of Attention produces more than twice as many peaks in the product part of the ad, particularly when images are on screen that activate our basic senses such as taste, touch or smell.

Our hypothesis for this is that these are images that tap into the oldest, deepest parts of the brain - the “reptilian” brain or amygdala, which sits in the anterior temporal lobe and which is where our primitive drives such as hunger or sex originate. Because these parts are deep in the center of our brains they’re harder to read with an EEG that picks up more electrical activity closer to the surface. In fact, that’s a major reason why some neuroscientists are using fMRI machines to pinpoint the deeper activity centers of the brain on advertising. In other words, there is still much to be learned about how advertising works in the brain.

Quite simple

Our overall conclusion from performing this experiment is quite simple. The best way to discover how your advertising is working with the consumer is a lot like the secret to a successful marriage: a man can learn something important by passively studying his wife’s behavior and non-verbal responses, but for the best results we also recommend talking to her.