Editor’s note: Dr. Simeon Chow is vice president, director of intellectual capital, at New York-based Audits & Surveys Worldwide.

A conflict exists between the ability of traditional advertising testing methods to provide creative guidance and diagnostic information while at the same time aiding management with the crucial decision of whether or not to use the advertising. This problem has stimulated firms to seek methods capable of helping both the development of advertisements (concurrent-development research) and the determination of which of several executions should be used in a campaign (post-development testing). Toward this goal, we report the development of a measurement approach and model designed to estimate the impact of variations in advertising copy on beliefs, attitudes, and purchase intention.

This measurement approach is guided by three important considerations: (1) the constraint of extending standard copy-test methods without needless complexities or burdensome added cost; (2) the necessity of grounding our procedures in well-developed, hierarchy-of-effects communications theory; and (3) the ultimate goal of providing managers with an actionable copy assessment procedure which provides richer diagnostic information than more traditional techniques. The primary benefits of this new approach are its diagnostic insights by:

  • providing creative guidance for the improvement of future copy;
  • serving as an aid to judgment for a go/no go decision on copy; and
  • relating findings to the overall advertising strategy.

Our copy-testing method, call SEQUENCE (Structural EQUations Estimation of New Copy Effectiveness) extends previous advertising copy-testing research by permitting an assessment of the strength of the linkages among brand beliefs, brand attitudes, and purchase intention without additional data-collection costs. Since comprehension of ad copy is routinely evaluated, and attitude measures taken, we envision this model as a low-cost addition to many standard copy-testing procedures used by advertisers and agencies.

The effects of advertising

Ad testing serves as a check of whether creative executions of advertisements are "on strategy" - that is, that they are capable of producing the communication effects that will achieve the communication objectives and positioning for the brand. Recall (brand awareness) measures have generated controversy over the years and, as a result, are not as influential as they once were. Recall and persuasion are conceptually two very different kinds of advertising effects, and one should never be used as an automatic proxy for the other. Thus, ads are often tested separately for these two effects. However, many in the advertising research community have concluded that persuasion-based measures of ad effectiveness are superior to traditional recall measures.

If persuasion is the desired outcome for effective advertising, then there are five communication strategies possible:

  • introduction of new salient criteria used to evaluate brands within a product class;
  • changing the saliency of brand beliefs based on existing evaluative criteria;
  • changing the strength of these brand beliefs;
  • changing the strength of the linkages among brand beliefs, brand attitudes, and purchase intention; and
  • changing beliefs concerning competing brands.

The proposed methodology evaluates all of the above possible communication strategies. The PACT principles (PACT Agencies, 1982) emphasize that sound copy-testing methods should be firmly grounded in communications theory. In the SEQUENCE Model, brand beliefs are viewed as the building blocks of brand image and preference. In order to affect changes in attitude toward a brand, it is necessary to change something about consumers’ belief structure with respect to the brand. Therefore, focusing on changes in global measures of brand attitude alone is insufficient to permit an assessment of how or why the observed changes were obtained. Anchoring on global measures of attitude as indicators of success retards the learning process of advertisers and agencies with respect to the reasons for differences in the effectiveness of alternative ad executions.

Our model is also consistent with DAGMAR guidelines (Colley, 1961) in that observed communication effects (not exposure or frequency) are evaluated systematically against explicit, communications-oriented, advertising goals. Colley argues that advertising results are best assessed within a hierarchy-of-effects theoretical framework and must be evaluated against advertising goals using benchmark measurements developed prior to implementation. SEQUENCE supports the use of multiple-item measures in a hierarchical framework, thus permitting explicit recognition of how random measurement error may (1) attenuate the precision of estimators and (2) reduce the power of statistical tests of hypotheses. The basic steps in the SEQUENCE copy assessment procedure are depicted in the flowchart.

SEQUENCE assumes the establishment of actionable, communications-oriented advertising objectives and the specification of advertising strategies designed to reach these objectives. At least one execution of the strategy is then developed and submitted to the SEQUENCE testing procedure. In the above figure, two alternative executions are compared to a no-exposure control group; however, more alternatives could have been tested. After exposure to the alternative ads (or without exposure in the case of the control group), multiple measures of brand beliefs, brand attitudes, purchase intentions (and ad attitudes if desired) are collected. These data are then subjected to two types of analysis. First, similar to traditional copy-testing methods, analysis of variance is used to examine differences in the effectiveness of the executions for changing beliefs, attitudes, and intentions. Second, structural equations estimation procedures are used to assess simultaneously the relationships among brand beliefs, brand attitudes, and purchase intentions for each experimental group.

Based on an overall evaluation of the ability of the competing executions to shift beliefs, attitudes, and intentions, and to modify the salience of key brand beliefs in a direction consistent with strategy, a go/no go decision is made. Insights gained from these analyses provide feedback for future copy-development efforts. In sum, we propose an approach to advertising-stimulus measurement and assessment that assumes a learning process and draws heavily on well-established multiattribute attitude theory. Recent advances in covariance structure modeling are applied to evaluate simultaneously the reliability and validity of our measurement model, the strength of modeled relationships, and the overall goodness of model fit.

Repositioning a toothpaste brand: a case study

In this section, we describe a straightforward application of SEQUENCE to the toothpaste industry. For a more technical example and explanation on how specific advertising effects are considered, see Chow, et al (1992).

The advertiser has identified two salient toothpaste benefits, one a cosmetic benefit (tooth whitening and breath-freshening) and the other a protection benefit (cavity prevention and tartar control). The advertiser desires to reposition the brand known for its cosmetic effects as a toothpaste that is also superior with respect to the protection benefit. A successful advertisement should lead to an improvement in average belief strength with respect to protection and a strengthening of the relationship between beliefs about the brand’s protection characteristics and attitude toward the brand.

Method

Measures taken from one group after exposure to an execution were compared to the reactions of a control group of people who were not exposed to the ad. Differences on the measures are ascribed to the effects of exposure. A shopping mall intercept sample of 350 respondents was obtained. Two hundred respondents were exposed to the ad, while 150 respondents comprised the no-exposure control group. Likert scales were used to capture the extent of a respondent’s agreement with the following belief statements: (1) keeps breath fresh for hours (Fresh), and (2) provides maximum protection against tooth decay (Protect). Brand attitude was measured on a single five-point scale anchored by excellent-poor. Purchase intentions were captured on a single five-point scale anchored by very likely-very unlikely.

Step 1: Test for differences in mean belief strength on the Fresh and Protect beliefs. Differences in mean attitude and purchase intentions scores between the execution and the control group were also tested.

Step 2: Estimate a model with Fresh and Protect as antecedents of brand attitude and brand attitude as the sole predictor of purchase intentions for each of the two groups. Direct comparisons via chi-square difference tests of the strength of the linkages between the variables are made. The objective of these statistical tests is to determine whether or not the ad affected the salience of the target belief, Protect. This is accomplished by comparing the magnitude and the sign of the path estimates between the target belief and attitude for each of the groups.

Step 3: Check to determine if the ad had any unexpected negative effects on the strength or salience of the other brand belief, brand attitudes, or purchase intentions.

Results

As desired, mean belief strength for Protect was significantly stronger for the ad group than the control group (mean = 4.06 and 3.83, respectively; p < .01, one-tailed test). Further, brand attitudes (ad mean = 3.47, control mean = 3.18; p < .01) and purchase intentions (ad mean = 3.33, control mean = 3.10; p < .05) were significantly enhanced by exposure to the ad. The other brand belief, Fresh, was not affected by ad exposure (p = .28). These results suggest that the ad effectively communicated the ability of the brand to prevent cavities while not detracting from beliefs regarding its cosmetic properties.

Traditional copy testing procedures often stop with this comparison of means. However, the ad may have had other effects on respondents beyond this observed shift in belief strength. In particular, we would hope that the repositioning strategy also affected the salience of the targeted belief. However, an ad which successfully enhanced belief strength regarding a targeted benefit but which also reduced or eliminated the salience of that benefit to brand attitude would accomplish little. Therefore, it is necessary to test changes in benefit salience, as well as mean shifts. In SEQUENCE, this is done by simultaneously estimating the model described previously using a structural equations estimation procedure. Without complicating the example, we have omitted the description of fit statistics. We note, though, that the model fit the data well.

We begin by examining the effect of ad exposure on belief salience. The key parameters are the regression coefficients reflecting the strength of the relationships between beliefs, Fresh and Protect, and brand attitude. First, in the no-exposure group, the influence of Fresh and Protect on brand attitude was estimated to be roughly equal (b = .411 and .370, respectively). In the ad group, the influence of Protect on brand attitude was enhanced (b = .542), a result which is consistent with the advertiser’s strategy, while the influence of Fresh on brand attitudes remained essentially unchanged (b = .454). A chi-square difference test confirmed that only the increase in salience for Protect was statistically significant (c2(1) = 5.59, p < .05).

In summary, SEQUENCE analysis revealed in a copy test that the advertisement successfully enhanced consumer beliefs regarding the cavity protection afforded by use of the toothpaste brand. In addition, the salience of the protection belief to brand attitudes was enhanced significantly without adversely affecting brand attitude or purchase intentions. Thus, the path analysis results, in combination with the results of traditional tests of mean differences, provide strong support for the efficacy of the ad.

Reading too much

Through SEQUENCE, the decision on which execution, if any, to adopt is made with more complete information and would depend on the advertiser’s evaluation of the total package of effects observed in comparison to a control group. A common problem is reading too much into a copy test result by seizing on one or two multiple comparisons, as is commonly done with ANOVA results. Our example is very straightforward, with only two belief items. However, in many copy tests, a large battery of belief items are assessed by the respondents. The likelihood of falsely concluding that a significant difference exists when it does not (Type I error) increases when multiple comparisons of mean differences are possible and there is no recognition that a certain proportion of comparisons will be significant by chance. This problem is lessened by accounting for all the effects of beliefs simultaneously in a structural equations framework.

SEQUENCE is designed to help management evaluate new copy alternatives before placement. The system is intended to: (1) rapidly predict a new copy’s effect on brand beliefs, attitudes, and purchase intentions; (2) produce actionable diagnostic information that can be used to improve the copy execution; and (3) permit evaluation of alternative copy executions. We believe the procedure provides more information with respect to the evaluation of advertising copy than traditional approaches at about the same costs; therefore, its diagnostic value is high.

References

Chow, Simeon, Randall L. Rose and Darral G. Clarke. "SEQUENCE: Structural Equations Estimation of New Copy Effectiveness." Journal of Advertising Research 32, 4 (1992): 60-72.

Colley, Russell H. Defining Advertising Goals for Measured Advertising Results. New York: Association of National Advertisers, 1961.

PACT Agencies. "Positioning Advertising Copy Testing." Journal of Advertising 11, 4 (1982): 3-29.