Editor’s note: David Langley is director of strategic research and analysis, and Lori Cook is a senior project manager, at Blue Cross Blue Shield of Maine, Portland.

This article is the second in a three-part series designed to provide real-world business examples of the effective use and application of research and statistical tools for supporting resourcing and priority-setting decisions. These research application issues have arisen through the authors’ work with regional and national studies in health care and other industries. Each of the three articles in the series provides a summary review and example of how marketing research, when approached as a credible discipline and with a clear view of specific decision support needs, can very effectively inform executive decision making.

The first article in this series, “Effective uses of ‘effect size’ statistics to demonstrate business value,” appeared in the October 1999 issue of Quirk’s. The third article, “Impact of health on satisfaction,” slated for June 2000 issue, will discuss an example of an effective approach to segmenting the customer base for refining satisfaction-based initiatives and related resource allocations and decision-making. It will include a review of the technique’s business value and use in decision making along with discussion of how operational and survey-based data can be integrated to demonstrate the validity and practical nature of customers’ responses on surveys.

When advertising, promoting, and testing brand concepts, how can we ensure that we are consistently delivering a clear and understandable message to our advertising audiences and our survey respondents? In one company’s recent experience, apparent contradictions in consumers’ perceptions of brand concepts - as observed in responses to survey questions regarding selected brand attributes--raise a number of challenges and questions in this area for both researchers and marketers. For the researcher, issues raised include those of survey methodology (e.g., question wording, order, and scales) as well as the analysis and uses of survey data. For the marketer, the presence and magnitude of these contradictions emphasize the importance of message intent and clarity when advertising and promoting the brand, particularly among key target segments.

The appearance of contradictions

Respondents to a 1999 advertising effectiveness study were asked to rate each of five key brand attributes at two points during the course of a telephone survey interview. In the first series of ratings, near the beginning of the survey interview, the attributes were presented as statements describing the company; respondents were then asked to rate their agreement or disagreement with the statements on a five-point Likert scale:

Example: “On a scale of one to five, where one means you strongly disagree and five means you strongly agree, how strongly do you agree or disagree that [company] offers a best value in [product]?”

In the second series of ratings, presented toward the end of the survey, the attributes were included among a list of 20 adjectives (the order in which the adjectives were presented to respondents was randomly rotated). Respondents were asked to dichotomously rate (i.e., yes/no) whether each adjective described the sponsor.

Example: “The following are a list of features which may or may not describe [company]. Please indicate which features you feel describe it. . . Best value in [product].”

When analyzing data from the survey, contradictions were discovered between how respondents rated the brand attributes in the first series of ratings and how they rated the same attributes in the second series of ratings. For example, some respondents who “agreed” (i.e., a top-two box rating) with a statement in the first series of ratings disagreed (i.e., a “no” response) that the adjective described the company in the second series of ratings. Conversely, some respondents who “disagreed” (i.e., a bottom-two box rating) with a statement in the first series of ratings agreed (i.e., a “yes” response) that the adjective described the company in the second series of ratings.

In further analysis conducted to determine the extent of these contradictions1, it was determined that contradictory responses occurred among 45 percent of the respondents (i.e., 45 percent of respondents had a least one contradictory response when rating the five attributes).

At the attribute-specific level, the percent of contradictions ranged from 11.3 percent to 17 percent (see Figure 1).

Figure 1

It was also determined that the direction of the contradiction (e.g., positive ratings in the first series shifting to negative or “don’t know” ratings in the second series, vs. negative ratings in the first series shifting to positive or “don’t know” ratings in the second series) differed depending on which attribute was being rated (see Table 1). Attribute A (“value”) ratings were more likely to shift from a positive rating in the first series to a negative or “don’t know” rating in the second series. The direction of contradictions was more or less evenly distributed for Attribute B. Among Attributes C, D, and E, ratings were more likely to shift from a negative rating in the first series to a positive or “don’t know” rating in the second series. For all attributes, a substantial portion of the contradictions occurred when a “don’t know” rating in the first series shifted to a positive or negative rating in the second series.

Table 1

“Contradictors” systematically differ from “non-contradictors”

Further analysis was conducted to ascertain whether there were any systematic differences (e.g., attitudinal, demographic) between respondents with contradictory responses and respondents with consistent responses2. The presence of systematic differences between these two groups would suggest that the occurrence of contradictions should not simply be attributed to random “noise” or measurement error. A number of significant differences were found.

Attitudinal indicators: lower ratings by “contradictors”

Responses of the two groups were compared on the items that - including the five attributes - comprise the 20-adjective series that was presented to respondents at the end of the survey. On all 20 items, the ratings of the “contradictory group” were significantly lower (p <= .05) than the ratings of the “consistent group.”

Responses of the two groups were also compared on four satisfaction measures that came in between the first series of ratings and the second series of ratings. These items measured overall perception of and satisfaction with the company. Again, the ratings of the “contradictory group” were significantly lower (p <=.05) than the ratings of the “consistent group” on all four measures (see Fig. 2).

Figure 2

Demographic differences: unique contribution of gender

The two groups were compared on income, age, education level, region, customer status (i.e., customer vs. non-customer), and gender.

While the two groups did not differ on income, age, education level, and region, non-customers were somewhat more likely than customers to have contradictory responses (32.4 percent of non-customers compared to 26.8 percent of customers).

In addition, women were more likely to have contradictory responses (32.8 percent of women compared to 27.3 percent of men)3. This gender difference cannot be accounted for by other attributes related to “contradictors.” That is, while contradictors tend to have less favorable views of the company, be less satisfied with the product, and be non-customers, women do not necessarily share these same features (i.e., they are not more likely than men to be non-customers, nor are they are systematically less satisfied). Therefore, it can be assumed that there is a unique relationship between gender and the likelihood of contradictions occurring.

Table 2

Other gender differences include the following:

  • Women were more likely than men to shift from a “don’t know” to having some opinion (for example, for Attribute C, 32.5 percent of women who contradicted themselves shifted from a “don’t know” to a “yes” or a “no” response, compared to 18.9 percent of men).
  • Women were particularly more likely than men to have contradictory responses for Attribute A (“value”): 15 percent of women compared to 10 percent of men had contradictory “value” responses.
  • For the “value” attribute, women were also more likely than men to shift from a negative to a positive response (61 percent of women compared to 49 percent of men), while men were more likely than women to shift from a positive to a negative response (23.5 percent of men compared to 16.7 percent of women).
  • On the 20-adjective list, women were consistently more likely than men to give “don’t know” responses.

Unintentional learning

Analysis of the data points to systematic attitudinal and demographic differences between contradictors and non-contradictors. As noted previously, contradictors have less favorable views of the company, lower levels of satisfaction with the product, and are somewhat more likely to be non-customers and women. These findings suggest that these contradictory responses cannot simply be attributed to measurement error.

It is hypothesized that during the course of a survey interview, there was a degree of “unintentional learning” for the respondent/consumer; that is, through presentation of the survey questions that followed the first series of brand attribute ratings (i.e., the four satisfaction measures and the 20-item adjective list) respondents obtained and thought about “new information” to define and interpret the brand and its attributes. This new learning was then reflected in the (sometimes contradictory) attribute ratings at the end of the survey.

“Value” attribute

Although the Attribute B through E contradictions are interesting and of some concern, primary attention needs to be turned to Attribute A (“value”) contradictions. For researchers and marketers, the importance of the finding of contradictions related to this attribute is due to the following:

a) the brand attribute “value” plays a critical role for competitive advantage and brand strength;

b) consumer-based testing used as a basis for the tested advertising design has articulated the importance of this attribute to purchase decisions.

As noted earlier, one-in-six of surveyed consumers held contradictory views of the value attribute (Attribute A). Over half of these value-contradictors moved from a positive to a negative position during the course of the interview. The overall tendency of ratings to move from positive to negative suggests the presence of problematic “unintentional learning” about this attribute during the course of the survey interview.

Key market segments

Another area of interest is the higher likelihood of contradictions in key market segments.

Gender
As noted earlier, findings related to this market segment include the following:

  • Contradictions were more likely to occur among women than among men.
  • Women were more likely than men to shift from a “don’t know” response to having some opinion.
  • Women were particularly more likely than men to have contradictory responses for the value attribute.

The importance of these findings relates to the role of women as key decision makers in the market area represented by this advertising study (i.e., health care). The findings suggest the following considerations for this market segment:

  • Women may be giving higher levels of consideration to attitudinal ratings.
  • Women, as the key decision makers, may be demonstrating a higher level of responsiveness to new information and learning in this area, particularly as it relates to value.

Customer status
Contradictions were more likely to occur among non-customers than customers; this suggest a higher degree of unintentional learning among market segments who lack experience with the company.

Conclusions

Interpretations and considerations being used by for advertising and research development include:

  • There are identifiable segments in which consumers’ understanding and perceptions of the value attribute are not clearly defined but are able to be informed through information and learning.
  • Utilizing the ability to move and shape consumer opinion through the presentation of information is a standard opinion survey practice: this can be readily leveraged to manage unformed value perceptions.
  • In the survey development phase of advertising research, careful consideration needs to be given to the clarity of question wording and the potential impact of question placement on responses.
  • Non-customers: value perceptions among non-customers (who lack experience with a company or brand) are less defined than those of customers; these perceptions and understandings can be readily shaped through information and intentional learning.
  • Particular consideration can be given to the design of value messages for women; in health care markets, for example, this segment has a primary role as decision maker; the higher likelihood of women to move from a “don’t know” to an opinion position (positive or negative), suggests a higher level of responsiveness to new information than for men, in this case.

Taken together, these considerations point to the need to think carefully about “value” concepts (as well as other core brand attributes) when designing messages, advertising, and survey questions. Identifiable consumer segments are particularly sensitive and responsive to new information and opportunities for learning. Since “value” concepts are reinforced by experience with the company and its products, building the value dimension of the brand among non-customers requires added attention by the advertiser. Likewise, emphasizing the informational needs and learning styles of selected segments (e.g., the role of women as primary household decision makers regarding health care) is a key area of focus for effectively tailoring brand messages and their assessments.

Notes

1 For the purposes of this analysis, a “contradiction” was defined as follows: a) a positive rating on the first series (i.e., a top-two box agreement rating) followed by a negative rating (i.e., a “no” response) or a “don’t know” response on the second series; b) a negative rating on the first series (i.e., a bottom-two box agreement rating) followed by a positive rating (i.e., a “yes” response) or a “don’t know” response on the second series; or c) a “don’t know” response on the first series followed by a “yes” or a “no” response on the second series.

2 Since planned minor question wording differences for Attributes D and E offer a possible explanation for the contradictions found in these attributes, the definition of the “contradictory” group was limited to respondents with contradictions in Attributes A, B, or C.

3 p <=.1.