Is it live or is it Gold'n Plump?
Editor's note: Tim Huberty is vice president, director of research and account planning, at the Clarity Coverdale Rueff advertising agency in Minneapolis. He also teaches at the Graduate School of Business at the University of St. Thomas in St. Paul.
Over the past few years, there have been many technological breakthroughs in testing advertising copy. Unfortunately, no matter how impressive many of those systems are, they are often too expensive for smaller agencies and clients with moderate budgets. In fact, testing a campaign using one of those systems can be more expensive than the media schedule for many clients.
Consequently, those of us at smaller agencies just have to be a little smarter and make our research dollars stretch a little further.
The challenge
Like many companies, Gold'n Plump Chicken, a regional processor and marketer of fresh chickens in St. Cloud, Minn., found that its competitors were getting increasingly aggressive. Many years ago, Gold'n Plump Chicken had successfully branded what had been considered a commodity product, but now realized the risk of consumers being seduced by lower priced competitors. Gold'n Plump recognized the importance of protecting its dominance as a premium price, higher quality product. Unfortunately, Gold'n Plump also had a limited marketing budget.
It's hard to find a point of difference in what is often seen as a commodity category. However, previous research had shown that Gold'n Plump was perceived by consumers as the more "thoroughly cleaned" chicken. Consumers were surprised and disappointed to find competitive chicken had extra fat or skin "hidden" under the chicken in the package. This was true of cut-up chicken as well as whole. In the previous research, consumers accepted competitive chicken for what it was - a product of "okay" quality. At the same time, they perceived Gold'n Plump as a better, cleaner chicken. The challenge was to communicate Gold'n Plump's unique difference to consumers and get them to define and to demonstrate exactly what "thoroughly cleaned" meant to them.
Tweaking the methodology
Using that simple phrase, the creative team developed several executions. A few weeks later, 10 ideas were shown to Gold'n Plump in very rough form. In this case, that meant the client was shown rough sketches torn from a yellow legal pad. The agency and client eliminated seven of the 10 executions, but could not decide which one of the remaining three would be most effective in communicating "thoroughly cleaned" to consumers. Consequently, it was decided to ask them.
Both client and agency had been long-time believers in a forced exposure method of copytesting. However, with three executions and little money for testing, some tweaking of the established way of copytesting was necessary.
Neither client nor agency had the time or the money to produce expensive testing materials. In the past, respondents had been shown actual storyboards and probed for their reactions. One problem with testing storyboards was that the interviewer was relied upon to "walk" the respondent through the commercial. Our own validation studies had shown that storyboards presented by "livelier" interviewers garnered higher scores. In this case there were no storyboards, but there were rough sketches - and a creative team. Putting those two together, videotapes were made of the copywriter explaining the sketches. Three separate videotapes were made, all using the same presenter. Each video was approximately two minutes long.
Next, a supplier was chosen to collect the data. Although the supplier had done excellent work in the past, we requested that they assign their "best" interviewers to the project. To further ensure the highest quality of data collection, the agency's research director went on-site and personally trained the interviewers. He stayed on-site for the first two days of interviewing and critiqued each interviewer after her first few sessions.
Aversion to numbers
Creatives have long had an aversion to numbers. To overcome this negativity, quantity was traded for quality. Instead of showing the videos to 100 people and having them rate statements or give short, five-word answers, it was decided to slash sample size, but collect in-depth, meaningful answers. Interviewers were encouraged to probe extensively. Instead of doing hundreds of interviews, 90 25-minute "sessions" were conducted in which each of the three videos was reviewed by 30 people.
To understand how each concept was working and to obtain information which would be of maximum benefit to the creative team, respondents were urged "get in touch with their feelings" and describe how each commercial affected them. After all, one verbatim is worth 100 numbers.
In-depth verbatims had been used at our agency to impact creative development for some time and had always been very helpful. Somewhere along the line, we had begun to take photographs of the respondents. When presenting the research results to the creatives, these photographs would be shown to them with the explanation, "These are the people that we talked to, these are the people who you are going to be writing to."
At the same time, we doubted the interviewers were capturing everything the respondents said exactly as they said it. Further validation studies demonstrated that interviewers often missed at least 50% of what the respondents said. No interviewer can write as fast as a person can speak. Furthermore, even if an interviewer could write that fast, the interviewer could never capture the emotion of what the respondent was saying.
Since videotaping had worked so well at the front end, we decided to try it at the "back end." A video camera was set up at the field service and about one-half of the interviews were taped. The interviewers still collected the data, feverishly writing down all that was said, but behind them a video camera also recorded the interview. Afterwards, a three-star rating system was used to evaluate each interview. This rating system wasn't used to indicate how much respondents liked the executions, but rather to show how articulate, imaginative and thoughtful they were. Now, there were numbers, verbatims and a video record of respondents talking about the three creative concepts.
In-depth analysis
When the interviewing was completed, the agency's research department coded and tabulated all questionnaires. Concerns about the objectivity of an ad agency evaluating its own creative were mitigated by the first-hand understanding and intimate knowledge the agency had of the project.
Initially, the closed-ended numerical ratings for each of the three concepts were very similar. Purchase intent and attribute ratings were amazingly identical. That's why the in-depth probing became so essential.
With one person coding and tabulating the results, a common basis of comparison was established. It was simple to compare the strength of the open-ended executional and message elements across each concept. Again, they were very similar for each of the three concepts. However, what really separated each of the concepts were the secondary messages that had been generated from each concept via the in-depth probing.
As expected, in response to all of the concepts, respondents talked about how "thoroughly cleaned" meant that Gold'n Plump had the "excess fat removed," "no pinfeathers" and was "already trimmed." However, in-depth probing uncovered that one concept had also communicated to twice as many people that "thoroughly cleaned" meant that Gold'n Plump chicken was "convenient":
"I thought it was a good presentation. It made you think about the chicken you bought in the past and how much cleaning and preparation it took and compare it to the Gold'n Plump Chicken. This is more convenient to fix."
The in-depth probing during the interview "sessions" also revealed that this particular concept - in which a hungry fox chooses a package of Gold'n Plump Chicken over a live hen standing next to him - was communicating that Gold'n Plump was "Fresh - Good as live chicken." When the concept was conceived, nobody had ever imagined that this message would be communicated:
"The fox is hungry, sees a live chicken. It appears he will eat the chicken. Then the hand reaches in with Gold'n Plump chicken. This causes the fox to decide what chicken he wants. He decides he wants the chicken that is already cleaned up. 'Gold'n Plump - it's as fresh as live chicken.' "
Finally, from this one execution, many more consumers took away the idea that Gold'n Plump chicken was also healthier. Again, this was not one of the messages that was originally intended to be communicated (in fact, it was never even imagined):
"It is hassle-free. It saves you lots of effort and you don't have to worry about a lot of waste. It's convenient. It's fresher, therefore it's healthier. A fox, being a clever animal, knows a good deal when he sees one."
Actionable results
When the numbers were tabulated and the verbatims were typed, the results were presented to Gold'n Plump and the creative team. All were shown clips of respondents defining exactly how each of the three concepts defined to what extent Gold'n Plump was the "thoroughly cleaned chicken." The written report contained over three times as many pages of verbatims as written tables.
The television commercial, "Fox and Hen," was produced and ran in several markets last fall. However, in addition to the television commercial, a totally integrated marketing campaign was born, as magazine ads, in-store posters and even outdoor boards were produced, demonstrating Gold'n Plump's unique selling proposition.
Follow-up telephone tracking revealed a significant increase in advertising awareness for Gold'n Plump Chicken. More importantly, despite the fact that the television campaign ran only three months, attributes scores of Gold'n Plump's competitors actually declined.
A lot of research time and effort went into the project, but the on-site monitoring of data collection and hands-on involvement during data analysis identified which commercial performed at a superior level. The results prove that you can stretch your research dollars and still do great work.