Editor's note: Terry Lawlor is EVP product manager at marketing research software firm Confirmit, London. 

The shift to digital experiences continues, with more digitally-savvy youngsters gaining purchasing power every day. Marketing researchers need to accelerate their initiatives to engage these consumers throughout the purchasing life cycle, from awareness generation to purchase interactions and into post-sale services. The hottest technologies in 2017 will combine video, mobile, social and location.

One of the most exciting areas of technology development in marketing research right now is facial and emotion recognition. A November 2015 study, How To Measure Emotion In Customer Experience, by Forrester stated, “How an experience makes customers feel influences customer loyalty more than effectiveness or ease in 17 out of 18 industries that we studied in Forrester’s Customer Experience Index (CX Index), U.S. Consumers Q1 2015. Over time, the emotions that customers feel can damage – or improve – their perception of the overall experience and the brand.”

Emotion detection

Marketing researchers are under increasing pressure to deliver value to brands. Adding to that pressure is an ongoing decline in survey response rates and challenges with collecting data from specific demographic groups. Harnessing emotions provides real opportunities to drive customer spending and enhance loyalty.

Understanding emotions is powerful but difficult to achieve. Facial expressions are strongly linked to emotions. When assessing emotional response, many researchers are looking beyond the option of human observation of recordings of consumer reactions. Facial expression recognition technology is beginning to deliver analysis without human observation bias. For example, emotion detection is being developed to detect feelings such as happiness, surprise, sadness, anger, disgust, fear and contempt – all types of emotions that may not come through in a survey.

The core use case for facial recognition technology is currently advertising analysis. This is particularly helpful when assessing ad performance. Client-side researcher teams turning to this technology may use benchmark analysis provided by companies such as Affectiva, an emotion measurement technology company that grew out of MIT’s Media Lab. But there are a multitude of uses for this type of technology, such as improving digital training by running an assessment on Web site videos to help pinpoint a few spots were confusion was registered.

Traditional methods meet technology

Technology will not replace your existing activities. Rather, look at it as an opportunity to include multimedia elements into a survey – for example, video – and apply another level of response analysis to that element. Emotion detection software simply adds to the toolkit available to the experienced marketing researcher. It may reduce the need for focus groups, but beyond that, it’s an addition, not a replacement.

Let’s look at the primary use case for implementing emotion detection: ad testing. Within a survey you can play your ad, during which time the respondent’s Webcam will record their reactions. Traditionally, respondents answer questions about the advertisement they’ve just been shown, rating it on various scales. A key issue here is that you rely on your respondent’s ability to recall what they’ve just been shown, their interpretation of their own emotions and their ability to put those emotions into words. Alternatively, researchers can film the interaction and employ people to observe and record the reactions of the respondents. This is a costly process with many challenges, such as achieving consistency and repeatability of results.

With emotion detection technology, an advertisement is shown within a survey and the respondent’s Webcam records their reactions. The technology monitors facial expressions as the respondent views your video, ensuring that the emotions are captured, even when the respondent is not fully aware of them. Using technology throughout the viewing stage enables advertisers to understand how the tiniest elements of their video may impact audience response.  

Video advertising research

The rise of video in online advertising will create a greater need for emotion detection software, as it requires innovative video analytics in order to understand how consumers respond to video in different scenarios, such as on YouTube, Facebook or Twitter, as well as through Web site browsing and more traditional kiosks and TV. By working to implement emotion detection, researchers will be able to harness video advertising research to better understand customers and what drives their behavior.

Capturing videos from consumers will also see accelerated growth in 2017. Whether for in-store research, mystery shopping, diary studies or simply sales and services feedback, video is able to bring people, their opinions and subject matter to life. As the technology advances, users will be able to quickly analyze the spoken word, actions, objects and sentiment, whatever the language of the consumer.

Location detection can add context to feedback and can be used to trigger research opportunities at key points of the purchase life cycle. Customer-facing businesses are expected to create dynamic shopping experiences that will encourage customers to return, either to browse and then buy online or to repeat purchase in-store. Using beacon technology, marketing researchers can selectively trigger in-the-moment actions, such as requesting customer feedback. This will dramatically improve their ability to understand key customer behaviors, such as path to purchase or non-purchase and promotion conversion rates.

These technologies provide researchers with unique insight into what impacts consumer awareness, emotions, intentions and purchasing behavior, while capturing valuable information that can help drive better business decisions, resulting in improved product and service offerings and more rewarding experiences. That said, researchers should carefully consider their target demographic prior to determining methodology, as many groups may be horrified at the idea of sharing their feedback by video, while others are more accepting. Or, for example, it may be an opportunity to capture the views of the increasingly important research targets such as Millennials and post-Millennials.

The future of emotion detection 

There is no doubt that new applications of software will emerge this year in both marketing research and customer experience disciplines – most likely with varying degrees of success. As with most advances of the last decade – mobile, social analytics, text analytics, beacon technologies and more – emotion detection will find its place.

Expect 2017 to bring further technology advances in relation to emotion detection, allowing people to gain insight based on emotional appetite or response instead of simply asking questions. We’ll see models being built around the concept of emotion capture – demonstrating how it is increasingly accepted as a valid business tool. It’ll be a few years before this really takes hold but we should see some major steps forward.

Further into the future? There are almost limitless applications of emotion recognition technology if we think outside the box. My personal favorite is the idea of using it in a retail environment. Imagine a world in which the faces of people waiting in line in a store are scanned and analyzed. Not only could we reduce the need for customer experience surveys about a store visit but retailers could identify staff who make customers happy or stores that perform well and conduct A/B testing on any number of elements. Spooky? Perhaps. But exciting!