Editor’s note: Will Headley is senior director, labs at BrainJuicer, a London-based research firm.
By the time you read this we will be in the future. Not just any old future but the future – the autumn of 2015 that Marty McFly travels to at the end of Back to the Future. Don't worry – this isn't another review of failed market research predictions from 30 years ago, nor a new set. It’s a simple set of tips on travelling back in time and how to avoid muddling your timelines when testing branded content and experiences … with just the occasional 1980s’ sci-fi reference.
You've almost certainly seen Back to the Future and recognize some of its distinctive assets. Like all the most famous brands and franchises, it has plenty of them: the DeLorean time-travelling sports-car; Michael J. Fox in double denim and red gilet; white-coated, shock-haired eccentric scientist Doc Emmett Brown; and perhaps the flux capacitor, the three-pronged pulsing heart of the time machine. The science behind the last of these is – naturally – never really explained but on further investigation there's something to it. “Flux” for a sparky flow of energy; “capacitor” as a place to store that energy for retrieval later on.
It’s well-named, because it turns out that's how all of us normally time-travel. What we experience in the present is a flux – and we revisit an experience by retaining some of it in the capacitors that are our memories.
Researchers have relied on the capacitors until recently, when a new generation of tools has given them access to the flux. But do they really understand which is important?
The man with two brains
Clearly our experiences – watching a film, viewing an ad, trying a new product or enduring pain – can shape our memories and future behavior. But there’s a problem with thinking of this as a neat, linear flow. As Daniel Kahneman, recipient of the 2002 Nobel Memorial Prize in Economic Sciences, has shown, the capacitor of the remembering self isn’t terribly faithful to the flux-y experiencing self.
The human brain is energy-hungry but thrifty (around 20 watts vs. the 1.21 gigawatts needed by Doc’s time machine) so it can’t and doesn’t store a flawless record of every experience. This creates heuristics (mental shortcuts), biases and fallacies aplenty for researchers and marketers to grapple with. A lot of stimulus we simply filter out and forget; we have no conscious memory of a particular thing we saw or heard.
This lowers the resolution of research but doesn’t mean people give outright wrong answers. More troublesome is our tendency to invent, adapt and re-weigh memories, which is why claimed ad recall has little to do with actual ad exposure. This is an old problem, one which traditional surveys have struggled to cope with – or turned a blind eye to – for decades. Requesting slow, reflective thinking from respondents long after an event certainly doesn’t talk to the experiencing self – that needs in-the-moment measurement. But traditional surveys may not even get much from the remembering self, as respondents grasp for acceptable, plausible or easy responses to impossibly difficult memory tests.
If you keep things simple and intuitive you can, like Kahneman, measure both the experiencing and remembering selves. He points out that these two are not analogous to his other famous metaphoric split of the mind, System 1 and System 2. Instead both selves are functions of System 1, the ancient, reflexive brain which uses shortcuts to make virtually all our decisions (unlike slow, calculating System 2, whose role is mainly to post-rationalize decisions and actions that have already been made or taken). Emotional as it is, System 1 isn’t all about the subconscious – even when we can’t control it we can usually still feel it. So experiments in this realm are feasible even without implicit and neurophysiological methods.
Rational thought and the remembering self
Kahneman’s most famous demonstrations of the experiencing vs. remembering clash involve pain. Colonoscopy patients were asked to rate their pain repeatedly during the process (experiencing self) and after they were asked to estimate the total pain experienced (remembering self). Another experiment had participants hold one of their hands in painfully cold water. The chilling experience was short and ended abruptly (hand removed immediately from cold water) or longer with a gradual end (water warmed slowly added by a degree before removing hand from it).
Most brands push pleasure over pain so I’ll add a happier example from my colleague, Tom Ewing: How to eat lunch the Kahneman way. For the experiencing self, it doesn’t really matter how you eat a sandwich. It will provide a fixed amount of pleasure regardless, in the same way its caloric value is fixed. That may vary per bite – perhaps you like the middle part best, less so the crusty, filling-free ends – but you get to the same sum of experienced happiness by the end of the sandwich.
If the remembering self worked rationally then the same would apply – the order of the bites wouldn’t matter, only the sum of pleasure across the whole lot. You’d forgive it not recalling every bite perfectly accurately but you’d hope it would store a reasonably representative average of the component bites. That would be helpful next time you went to choose a sandwich: Which of the options available has performed best on average against your internal norm of sandwiches?
Like the rest of our brains, though, the remembering self is not rational. It doesn’t give equal weighting to all bites of the sandwich, to all moments of the colonoscopy or cold water pain. It doesn’t even take a random selection of moments (which, averaged, would be similar to the experienced average) but instead it has a systematic bias, favoring certain moments over others. Specifically, it looks to the peak and end moments – the most intense part of the experience and the final moment. We store stories.
Packaged with this is duration neglect – the length of the experience isn’t given its due weighting by the remembering self. Twenty minutes of pain should be considered twice as bad as 10 minutes of the same pain level – but isn’t. So if you want to maximize the memory of enjoyment of your sandwich, you should save the best bit until last, make the peak and the end the same thing. Maybe (other) apes have figured this out – they tend to eat bananas upside down (or perhaps it’s humans who have them upside down?) starting with the yucky black bit of the banana rather than ending with it!
The same pattern applies to advertising and entertainment. Great books, films and videos maximize emotional payoff toward the end, not in the middle. Obviously jokes end with the punchline but it’s also wise for a stand-up comedian to end their set with their best joke, for a comedy night to conclude with the best comedian. This is what the remembering selves of the audience are most likely to take home and talk about, affecting their own and others’ behavior.
Perhaps it’s no coincidence that the best, most emotional ad we tested in 2014 was a joke. French brand Le Trefle’s Emma – starring a technophile man who wants his partner to use their tablet for everything – spends most of its 38 seconds on contempt and boredom as the tedious male lead berates his partner. Only in the final seconds does the brilliant punchline propel it to five-star levels of happiness. The experiencing self shouldn’t want to watch more of this sort of thing since netted over the 38 seconds it was pretty dull; the remembering self loves it though, shares it with their friends and buys 30 percent more papier from Le Trefle.
So does this mean the self that brands and research should worry about is the remembering one, not the experiencing one? Focus on the capacitor, not the flux? After all, the remembering self is the one who gets to travel into the future, buy stuff and tell his (somewhat fictionalized) story while leaving truthful experiencing stuck silently in the past. Yet as Behavioral Economics Guide Editor Alain Samson points out, there seems to be a trend in the opposite direction: Over the last 10 years Google Trends shows a big shift from interest in holistic “customer satisfaction” toward in-the-moment “customer experience.” This might just be semantics but if the shift in terminology is meaningful then it’s a major move away from measuring remembering toward experiencing.
Rationally this could be commendable – making more people happy across more moments, maximizing experienced utility – but in practical and behavioral terms it might be a mistake. The remembering self has more impact on future decisions, after all. And listening to only the experiencing self will tell you to avoid mistakes, be quick with unavoidable pain and slow and consistent for pleasure. On the other hand, the remembering self calls for the opposite with bad experiences – go slowly and carefully, avoid high peaks of pain, end gradually. Brands marketing to remembering selves might worry less about making occasional mistakes so long as they resolve them and delight with high peaks.
Neuro and biometric methods
Perhaps the trend toward measuring experience over memories is a practical one: Is it more feasible to get an accurate read of the former over the latter? In-the-moment reactions used to be hard to track but there has been a recent boom in exciting new methods aiming to measure the experiencing self more faithfully. These include implicit association testing (IAT), metaphoric scales, facial coding, eye-tracking, heart rate monitoring, galvanic skin response (GSR) electroencephalography (EEG), through to functional magnetic resonance imaging (fMRI).
Most of these interesting techniques are finding themselves classed under the broad umbrella of neuromarketing. It’s probably more helpful to use the tighter definition of neuroscience to mean only those which measure brain activity and the nervous system more directly. All of these require specialist kits and as such aren’t cheap, quick and easy to do on a robust scale. The king of neuro methods is fMRI, using expensive machines which, like the time-travelling DeLorean are super-cooled, high-powered, noisy and a little claustrophobic. They are hugely less mobile though – i.e., not at all.
In the recent past I was lucky enough to experience the power of University College London’s MRI machine. It doesn’t just allow a literal and fascinating glimpse into consumers’ (or in this case, co-workers’) minds but also detects brain activity down to an excellent spatial resolution of around 1mm3. It’s the way to see the amygdala in action, part of the brain which appears to have a crucial role in emotion and decision-making. UCL explained that the amygdala is sometimes called the brain’s “4F” nucleus, thanks to its role in feeding, fleeing, fighting and … er, reproduction.
So is fMRI the perfect tool for measuring emotional flux? Not quite. Its spatial recognition is terrific but its temporal resolution isn’t so good. Stimulus and response can’t be aligned second-by-second but only in longer blocks with breaks in between. It may, in fact, be better at predicting the remembering self, measuring memory encoding – both the amygdala and hippocampus appear to have important roles here. It may not be able to tell exactly what’s being written into the brain’s time capsule but it might still help explain behavior. A recent experiment led by Temple University (published August 2015) looked at a battery of new techniques and how well they could explain real-world advertising efficiency. FMRI added the most efficiency on top of traditional surveys (more than IAT, eye-tracking, EEG and biometrics).
While that’s an exciting finding, there are some clear practical limitations around context for fMRI. Everything must be done inside an MRI machine so any “experience” is a little cramped and limited to what can be streamed to them there like short videos, images, words, music, tastes and aromas. Even with large budgets, base sizes for fMRI-based market research are restricted to 10s, not the robust and representative bases of hundreds that most of us are used to for quant research. The nature of MRI means it’s highly unlikely to get significantly cheaper, easier or more scalable. So its best use may be in the testing and validation of more scalable methods, rather than as a routine method in its own right.
In Back to the Future, you may remember Doc’s failed mind-reading headset (or perhaps not as it is neither a peak of the film, nor at its end). This is a retro parody of EEG, a headset with an array of electrodes detecting the brain’s electrical activity. Unlike fMRI, EEG has excellent temporal resolution, not just second-by-second but millisecond-by-millisecond. It’s also more scalable and portable – if a little awkward – and it’s a more common tool in market research. The big problem is that it still doesn’t tell us what people are feeling (more if they are feeling) and spatial resolution is poor – it’s hard to be sure exactly which part of the brain the signals are from; deeper but crucial areas like the amygdala are especially hard to reach.
Away from pure neuro and biometric methods requiring specialist kit and facilities, there’s also been a boom in more scalable, online tools. There are many tech suppliers offering facial coding and eye-tracking where the only requirements are an Internet connection, a Webcam and some respondents. This form of eye-tracking might add an interesting diagnostic layer, telling you where someone is looking and for how long but it doesn’t tell you how – or even if – they feel about the experience. Does the ad’s protagonist bore, delight or disgust viewers? Really we need to know how they feel – which is the promise made by automated facial coding.
Most facial coding systems are ultimately grounded in the findings of psychologist Paul Ekman, who discovered that there are seven universal facial expressions for emotions, used and recognized globally: happiness, surprise, fear, sadness, anger, disgust and contempt. If you can train an algorithm in Ekman’s Facial Action Coding System (FACS), can a Webcam measure the experiencing self? It’s an exciting prospect. While most of the systems acknowledge the debt they owe to Ekman and FACS, very few are currently trying to measure all seven of the emotions he classified, let alone succeeding. Most commonly absent is contempt. There are various reasons for this. Some suggest that it’s essentially the same as disgust, so already covered (assuming you can identify that). True, it’s related to disgust but contempt has a unique expression as well as having different causes and consequences – indeed, it’s one of the most dangerous emotions for business, as it’s very hard to recover from.
Another problem is that “contempt” is not a common word – normal people rarely use it, few will ever mention it in open-ended questions or pick it from a text-only list. But one of the other promises of facial coding is that it gets at feelings which respondents can’t or won’t express verbally in traditional methods – all of us can feel, signal and recognize contempt even if we don’t know what it’s called. Automated facial coding requires the human participants to do the feeling and signaling, computer hardware and software to do the recognizing. With contempt in particular, the recognition part is proving to be a stubborn challenge. This is partly down to contempt being the only asymmetrical expression of emotion – it needs an especially clear, front-on view of the face as well as a more complex algorithm. Well-lit, lab-type conditions and well-behaved, motionless (but expressive) participants help but this hinders scalability and feasibility.
Luckily, while we wait for the algorithms to improve, there is a hack. You can bypass the need for the computer-based recognition part and play to the same respondent’s natural skill at emotional recognition. Indeed we find that once images of emotional expressions are included in surveys, not just text names, contempt is the most common negative response to marketing stimulus – so it can and should be measured. In the future, we hope to see automated facial coding improve, and maybe become the way to measure the flux of the experiencing self.
What about the remembering self? Perhaps in the future we won’t need surveys but today the right ones (ones which don’t force people into overly considered, System 2 decisions) remain the best way to talk to the remembering self. With the promise of tools measuring its experiencing counterpart, it’s tempting to talk down the remembering self. It might not be the more honest of the two (it’s prone to binning data, storytelling, self-delusion and losing track of time) but its ability to go back in time at all makes it uniquely helpful to marketers. When we make decisions, it’s the remembering self that provides the emotional triggers. Imagining the future and remembering the past are closely related brain functions so it’s worth chatting with the remembering self if you want to go to the future too.