Editor’s note: Marija Smudja is advertising insights director at EyeSee, Belgrade, Serbia.
In the world of digital advertising, there are dozens of tools that help marketers measure impact. However, not all measurements are equally useful. How do you reexamine advertising testing for the digital age and its distracted, advertising-avoidant consumers?
First, take a step back from the traditional testing routine: scrutinizing the ad creative through a focus group, displaying the stand-alone, full-length, full-size ad to a benevolent respondent and following it up with in-depth questions. Why? Because this process, aside from a delayed and conscious ad assessment, entails forced exposure of the tested material to the consumer. A critical precondition to evaluating ads is checking if they manage to reach the consumer and get noticed in its natural environment to begin with.
By using natural and unforced exposure of the stimuli, in an environment as close to reality as possible, we get an objective and accurate insight into the actual ad visibility. Studies done this way are the single most useful and authentic measurement today and are becoming a standard go-to approach for brands and companies doing ad pretesting. If you have a great ad that goes unnoticed when thrown into the highly competitive and crowded social media environment, you are wasting your money. The harsh truth is that the average consumer scrolls through their feed at a speed of around one-to-three posts per second, and the rule of thumb is an actual rule of the almighty scrolling thumb.
What type of context yields optimal results?
However, it is not only a matter of testing ads in context but also about what that context is. More precisely, is it as realistic as it can be? Which is better, testing it the way it would be perceived in real life – in the respondent’s own news feed – or in a natural, albeit simulated context?
Both approaches involve consumers being exposed to your ad in the digital clutter of images, posts, videos and outside distractions, exactly like they do in real life. Only, depending on the situation and the task at hand, you can pretest ads both by injecting the stimuli in the respondents’ actual social media feed or test them in a simulated feed.
What is ad insertion?
Ad insertion is a technique by which ads are placed into the real, live feeds or webpages, ensuring that the ad will be seen in a completely natural environment for the respondent. On the other side, a simulated feed is entirely created and controlled by the researcher, also allowing the insertion of the ads into desired places.
Both approaches have a specific set of KPIs – they measure the visibility of the ad (viewability) and time spent on the ad (attention). Each has its own advantages and drawbacks.
Ad insertion – actual news feed
Pros:
- Enables ad exposure in a completely natural environment – the kind of situation that it will be seen in by the viewers – which encourages the respondents to act normal, be completely relaxed and thus gives more precise data on ad performance.
- It is possible to place the tested ad in between any two posts on the timeline, and in this way measure not only its visibility but vary the position and measure the influence of its place on visibility and attention.
Cons:
- It is impossible to control the influence the feeds themselves have on post visibility. Every user’s feed is different, introducing an area that is not regulated by the researchers. The surrounding posts may have a specific type of content (videos, bright colors, engaging content) which can modify post visibility.
- Recruiting is potentially difficult. This methodology requires an application download and logging into the personal social media profile with their own e-mail and password. Although the researchers do not collect user data – no data is captured while they log in, the content manipulation is taking place inside an app and is not visible or affecting their profile outside of the app – a participant’s potential reluctance to share this information is understandable.
- The application of this methodology can have certain technical limitations and requirements since it is not accessed through a browser but as an app.
- Participants may or may not naturally browse faster after logging in, knowing that they are participating in a study.
Content management system (CMS) – simulated news feed
Pros:
- A simulated news feed enables an experimental approach to research or a study. As the ad is inserted into a preprepared feed, it ensures that the different KPIs – between two or three ads or ad variations – can be attributed to the creative direction of the advertisements, considering that all the other conditions are controlled.
- Because the researcher has full control of the environment in which the ad will be placed, there is room for exploratory research and play. This means a brand could measure the visibility of the advertisement in its surrounding as well as the effect and impact of the environment on visibility, leaving space for variations. Simulated news feeds allow you to carefully curate the environment, with a randomized combination of posts, a congruence between the theme of the posts and the ad, include various formats (video, image, gif, text).
- Similarly to the first approach, it is possible to place the tested ad on any spot in the feed, and in this way measure not only ad visibility but vary post position and measure the impact of the place and the order in which it shows up on the feed on visibility and attention.
Cons:
- A simulated feed is not a personal feed, which makes a full replication of the natural situation impossible, considering that in reality, the ad will be placed in many different feeds and profiles.
Measuring behavior
Is the respondent’s behavior genuinely different in a simulated vs. actual feed? My team ran a mini-study to uncover whether there are differences, and in which direction they are skewed.
We tested two ads for mineral water – one that promotes individuality and being in touch with nature (it opens with light colors, with a close-up shot of a face), while the other one puts a focus on family spirit and a sense of community (it shows a family in the opening sequence, in a slightly darker palette). Both ads were tested in real and simulated settings. The video ad was the sixth in order in the feed.
Let’s look at the results:
- Users scroll faster while logged into their personal Facebook feed. In a simulated feed, it takes participants around 22 seconds to reach the tested post, while in their own feed it took them around 20 seconds. This difference might be due to the fact that they are already familiar with the content of their feed, so they automatically skip/skim through the posts of certain people, groups or topics.
- Consequently, the visibility of the ad in the personal feed is slightly higher, but the time spent on the ad is lower. In a simulated timeline, respondents spend more time exploring the posts, because they are aware of the fact that they are part of an experiment and it feels less natural.
The ad promoting individuality and contact with nature performed better in both the personal and simulated feeds – a 10% higher visibility, and 0.5 seconds longer attention. The main takeaway: browsing is slightly different, but the conclusions remain the same. Testing in a CMS can provide high-quality insights that have a minimal absolute deviation from the data in personal feeds, but yield the same results.
Navigating the ad-avoidant consumer landscape
There is no good reason not to pretest your social media ads. However, in the age of cheap and fast solutions, the quality of the testing you do is vital. Using unforced exposure to check if the respondents have the willingness to watch or interact with the ad is indispensable for identifying thumb-stopping content. An environment that mirrors real life as close as possible will safeguard your brand from misleading or overly optimistic insights, and consequently, prevent waste of marketing funds. As the advertising scene becomes more crowded, settling for less-than-convincing testing environments will become a thing of the past. How close to reality will the conditions get? I don’t know yet. We have yet to see which of the two methods discussed here will dominate, but so far, CMS seems to be winning.