Research on Kaiser Permanente ads explores influence of language preference

There are Hispanics...and then there are Hispanics

Editor’s note: Jack Bookbinder coordinates advertising research for Kaiser Permanente, an Oakland, Calif.-based HMO.  Don Bruzzone is president of Bruzzone Research Company, Alameda, Calif. This article is adapted from a presentation made in March at re:think!2006, the Advertising Research Foundation’s annual conference.

The media habits of Hispanics differ widely. Measure their viewing and language preferences with one online research panel and you can get a very different picture than if you used other panels that recruit Hispanics or Latinos differently. This can have drastic effects on the outcome of your study.

This came to light as part of a massive two-year, 22,000-interview study conducted online to track the effectiveness of a $100+ million “Thrive” brand campaign by Kaiser Permanente, the nation’s largest HMO. It was the first national campaign with new agency Campbell-Ewald and its partner in Hispanic advertising, Accent Marketing. It started in the fall of 2004, and at this point has used seven different commercials, two of which are in Spanish, created specifically for the Latino market. Over 2,500 of those 22,000 interviews were with Latinos in California to study the impact of the Spanish-language advertising. It is that section of the ad tracking study that produced the findings that are the subject of this article.

Incidentally, we view the word Latino as meaning the same thing as Hispanic, with Latino being preferred in California.

The early waves

The study started with a baseline wave to show what the conditions were before the campaign started. Every six  months since then about 5,000 interviews have been conducted to monitor the main campaign’s effects. In addition to that, 600-900 Latinos in California were also interviewed.

In the Latino survey we intended to get at least 300 replies from Latinos replying in Spanish.

Figure 1 shows that goal was not achieved until the second wave. In the first wave we were surprised by a much lower response rate from those replying in Spanish. By Wave 2 we were able to locate more sources of Hispanic panelists who would reply in Spanish.

The importance of that difference was obvious from the start. We found those who replied in Spanish were more likely to recognize the Spanish-language commercials which Kaiser Permanente had been running on Spanish-language TV. Figure 2 shows that holds true for all three that had been tested by the end of Wave 2. (The first two were Kaiser Permanente Spanish-language commercials that had been running prior to the new Thrive campaign.)

Figure 3 shows how the percent that gave Kaiser Permanente a top rating in various attributes had changed by the second wave. For those replying in Spanish, these impressions had not improved. But for the group replying in English, the ones we assumed were more likely to have been reached by Kaiser Permanente’s regular advertising in English, there was some evidence of slight movement.

Wave 3

Since we had balanced the sample in Wave 2 in terms of language, our goal for Wave 3 was to see if we could increase the overall size of the sample and get more respondents replying in Spanish. The best opportunity for doing that appeared to be sending bilingual invitations to the types of panelists who had previously replied in English. Most indicated they were bilingual, so we wanted to see if we could get a significant portion to reply in Spanish. Figure 4 shows that the effort succeeded in getting a larger sample, but the bilingual invitations did not get more to reply in Spanish. In addition, the panel that had been our best source of Spanish-language replies in the first two waves, goZing, was bought out by Greenfield and was no longer available to us. In the end, the number replying in Spanish stayed about the same as in Wave 2.

There was a substantial increase in the percent who recognized “Avocado,” a Spanish-language commercial tested in both Wave 2 and Wave 3, among those replying in Spanish (see Figure 5). But there was no similar increase among those replying in English.

There was a much larger increase in the percent of Latinos responding in Spanish who gave top ratings to Kaiser Permanente (KP) attributes (see Figure 6). These increases were so large that based on increases seen in other tracking, there was reason to question the validity of the data.

The online panels

The nine online research panels listed in Figure 7 were used to obtain the Hispanic replies in Wave 2 and 3. The chart shows Luth was the only panel to be used in both waves, and roughly a third of the replies in both Spanish and English came from a new panel that had not been used before: Lightspeed.

Almost all of the increase in recognition of the “Avocado” commercial in Wave 3 was accounted for by the Spanish-language replies from that same panel: Lightspeed (Figure 8). Similarly, almost all of the improvement in overall impression and other attributes was accounted for by the very favorable replies that came in from that same panel that had not been included in previous waves (Figure 9).

Figure 10 shows viewing habits were significantly different between the panels, and this helped explain the differences in campaign performance.

Among respondents replying in Spanish from the Luth panel, a majority watch mostly English language TV. Among those from Lightspeed a majority watches Spanish-language TV at least half of the time, and the percent that watch mostly Spanish TV was more than double what it was for Luth.

But among those replying in English both panels were quite similar. A strong majority watched TV mostly in English.

Recruiting

What caused those key differences between the panelists who responded in Spanish? Figure 11 shows it was related to how the panelists had been recruited by their panels. All had Latino panelists who were recruited in English. They ended up in the firm’s Hispanic panel only because that is what they checked when filling out their demographic background. However, the goZing panel (later taken over by Greenfield) and the Lightspeed panel, which had produced higher recognition of Spanish-language commercials, had been recruited differently. Respondents had been recruited into the panels entirely in Spanish, from Spanish-language sites that in each case included the Web site of a major Spanish-language TV network. Both feel they know the language preferences of their panelists, always ask them to take a survey in their preferred language, and resist using bilingual requests. Further, both were the only panels that maintained Spanish-language Web sites that were used in their recruiting.

Wave 4

That gets us to the latest wave, Wave 4, conducted the end of last year. Given what had happened up to this point we had several goals:

Based on the earlier results we wanted to reduce the percentage of English-dominant respondents in the sample. We found they spoke mostly English at home, watched little Spanish TV, were less likely to be aware of health plan commercials in Spanish, and had awareness of other advertising that closely matched the rest of the population. They were less important in this tracking of Spanish-language advertising because Spanish-language media appeared to be reaching fewer of them.

To do this we set out to find more online panels that communicate only in Spanish with their panel members, and thus offer the survey in Spanish only, rather than a choice of language. This effort was quite successful: 79 percent of the 765 respondents in Wave 4 replied in Spanish.

But it was not as successful as we hoped in targeting viewers of Spanish TV. Many of these Spanish-dominant respondents watch a significant amount of English TV.

Wave 4 revealed that among those who usually speak Spanish at home, only a bare majority, 52 percent watch mostly Spanish TV. A surprising 33 percent spend most of their time watching TV in English. A relatively small number watch both about equally. That is shown by the bars on the left of Figure 12. It is as if a substantial number of Spanish-speaking Latinos are becoming acculturated to the English-speaking environment they are in, or are making a conscious effort to do so. Still, the largest number is comfortable spending most of their time watching TV in the language they use most. It is interesting that most fall in one group or the other. The number that watch both equally is quite small.

The pattern is noticeably different when you look at the results for the group in the center who speak both languages about equally. The percent who watch mostly Spanish TV is much lower. The percent watching TV in English is higher, and the percent that watch both types has increased dramatically to almost three times what it was for those who speak mostly Spanish.

Results for the third group of Latinos who speak mostly English show why we have been deemphasizing them in our Spanish-language ad tracking. When asked about their ethnic background this English-dominant group says they are Hispanic. But in terms of their viewing habits, almost all spend most of their time watching TV in English. The grey boxes across the bottom show that even though we had substantially increased the number of Spanish-dominant Latinos in the sample, 55 percent of all Wave 4 respondents still spent most of their time watching TV in English.

How do these viewing habits affect the number reached by commercials in Spanish? Figure 13 shows the results from this fourth wave for two of Kaiser Permanente’s commercials in Spanish that are called “Avocado” and “Hands.” It uses commercial recognition as the measure of whether they were reached. The number that remembered seeing them was higher among those who watch Spanish TV most of the time compared to those who watch mostly English TV. Recognition among those who watch both equally is in the middle.

We think the most notable finding on this chart is the 18 percent recognition of these commercials among those who spend most of their time watching TV in English. They have not said they spend all of their time watching TV in English. This suggests they do spend some time watching Spanish TV, and that time is enough for 18 percent of these English-dominant Latinos to have noticed the commercials. The difference in recognition - 18 percent vs. 30 percent - is far less than the difference in time spent watching the two types of television indicated by the previous chart (Figure 12). A substantial number noticed them even when the opportunities to see them were only a fraction of what they were for the heavy viewers of Spanish TV.

Here we think we have evidence of the type of “engagement” that is the subject of serious study by the Advertising Research Foundation. It suggests a substantial number of anglicized, English-dominant Latinos become engaged with commercials in Spanish, and remember them, even when they have had very little exposure.

Even stronger evidence of that is seen in Figure 14. It shows one way of determining if a commercial has actually had an effect on people who noticed it. Here we are looking at the percent who said they were very likely to consider joining Kaiser Permanente. It shows that figure for those who later indicated they recognized each commercial as one they had seen before, and compares it with the figure for those who didn’t recognize that commercial. It shows if there was a difference or “lift” in this indication of interest in joining Kaiser Permanente that was related to having seen each of four commercials. It shows this for the large group of Latinos who watch English-language TV most of the time. And, it shows this for four Kaiser Permanente commercials, two in Spanish and two in English.

What we found was a great deal of lift among those who had seen the Spanish commercials, but no consistent pattern of lift among those who had seen the English commercials.

Here we have even stronger evidence to back up the premise we alluded to previously. And it provides consistent evidence backing up a key premise other students of multicultural marketing have offered. Even the English-dominant, well-anglicized Latinos will react more favorably to an advertiser who shows some recognition of their culture.

Unaided awareness of the organization and its advertising is another measure of campaign impact. Figure 15 shows both are as high or higher among those who watch mostly Spanish TV.

Why those who watch both types equally were lower was not obvious. In fact it was very suspicious, so we looked into that further.

Suspicious findings

One of the first places we noticed some unusual differences between Wave 3 and Wave 4 was in the answers to a question asking if the respondent had ever been a member of Kaiser Permanente (Figure 16). It has an effect on answers, so it is something that is asked in almost all the organization’s surveys - and it is a figure that is known to stay fairly constant. But it didn’t stay constant here. Between the last two waves it dropped significantly among those who watch both English and Spanish TV equally, and it increased among those who watch Spanish TV most of the time. It suggested there had been a significant change in the make up of the sample between the waves.

More evidence of the same type was found in the number who said they didn’t know if they had ever been a member of Kaiser Permanente (see Figure 17). In Wave 4 it became very high among those who watched both equally - and the same pattern was seen for Kaiser’s competitors.

Similar evidence was found in the income question. Figure 18 shows how the percent that declined to answer also rose to unusually high levels among those who watched both.

Added a new panel

We again found most of these wave-to-wave differences came from the addition of a new panel, in this case it was Focus Forward. As shown in Figure 19, among the Focus Forward respondents 54 percent were in the “watch equal” category, the category that showed substantial differences since the earlier wave. Compared to other panels, the chart shows they were also more likely to watch both equally, speak both languages equally, and much more likely to say they didn’t know if they had been a member of Kaiser Permanente, and to decline to indicate their income level.

We added panelists drawn from this Focus Forward panel when we found they were one more panel that recruited in Spanish from Spanish Web sites, and we were impressed to find that was not the only way they recruited Latinos. They also post flyers in neighborhoods heavily populated by Latinos, use direct mail and advertise in Latino newspapers and church bulletins. This unusual recruiting produced other differences. Few reported above-average income. The percent from California’s central valley, where low-paying agricultural jobs are common, was greater than for the other panels. There was a more equal balance between men and women, and they were much less likely to have taken any other online survey during the past week. If nothing else, it broadened our coverage of a segment of the Latino population that was not as well represented previously.

 How do you explain?

But how do you explain the unusual results from this panel? After talking to experts, it is still pure speculation, but these are the best possibilities we have come up with.

• These respondents are more likely to speak both languages and watch both types of TV at home because they are still at the stage where they are struggling to find a place. They are not at the stage yet where they can afford to keep parents or others in the household who only speak Spanish.
• They don’t know if they were covered by Kaiser Permanente or any other health plan in the past because their coverage was likely to be handled entirely by their employer without their being aware of, or even being interested in, whose plan they were covered by.
• We can see several reasons for a large number at this level of the socioeconomic ladder declining to indicate their income. Many may have been embarrassed at how low it was. Others may have worked at many short-lived jobs during the year and honestly have no idea what their total income was for the year.

So, for the moment, we are treating these changes in our Latino sample as improvements in reaching a more complete cross-section of those who watch Spanish TV. But we are not sure about that. We are still working on it, and we would be greatly interested in comparing notes with others working in this field.

Final thoughts

Finally, what do we now know about the differences between panels and how they affect measurements of the effectiveness of advertising to Latinos? We’d summarize it this way:

• Advertising in Spanish-language media reaches a much larger share of those from Latino panels recruited in Spanish from Spanish-language Web sites, particularly the Web sites maintained by the major Spanish-language TV networks.
• In addition, Latinos are also reached by a substantial amount of the English-language advertising, but it doesn’t have as much of an effect on them as being reached by advertising in Spanish. That is particularly true for those who watch mostly English TV.
• Conclusions about the effectiveness of advertising in Spanish can be glaringly different depending on how Latinos are sampled.
• Inadvertently changing from one source of Latino samples to another can lead to jarring and seemingly impossible changes in the findings.