PBS station tests direct mail pieces with dial response measurement system

Dialing for dollars

If you're a fan of public television and/or radio, the words "pledge drive" probably strike fear into your heart. Pledge drives are an unfortunate necessity for public broadcasting entities, which depend on viewer and listener support to fund their programming and day-to-day operation. Audiences don't like them because they seem to go on forever and they foul up regular program schedules; the broadcasters don't like them because they know they're trying their audience's patience as they hammer away at them to start or renew their membership. In addition, it's difficult to find new ways to beg for money year after year.

For many stations, a less painful alternative or supplement to the pledge drive is the direct mail appeal. A PBS station in the Phoenix area, KAET-TV, uses a direct mail package to ask lapsed members to renew their involvement with the station. To pre-test the effectiveness of its appeals, the station late last year used MACH 2, a dial response system developed by Behavior Research Center, in four research sessions with groups of lapsed KAET members.

Dialing in

Like other group response measurement systems, MACH 2 allows each participant to privately give his or her response to a question by "dialing in" the answer using a hand-held controller. Marketers watching the proceedings are able to see the responses displayed in real time on a video monitor and add questions based on those responses. (See accompanying article for a more detailed explanation of group response measurement systems.)

The research was conducted by Behavior Research Center-Phoenix and The Direct Marketing Laboratory, Phoenix. Arnold Schwartz, president of The Direct Marketing Lab, says he saw pre-testing as a way to improve the direct mail process, which usually relies on response rates from test mailings to measure the success of a mail package.

"Although you know from your mail test whether you have a winner or loser, many times you don't know why, or what to do to improve your test. So it seemed to me that direct marketing had come of age to the point where it would be productive to pre-screen test packages prior to putting them in the mail, so that when you test in the mail the question isn't, 'Do I have a winner or loser,' but, 'By how much will it win?'

"Using the MACH 2 technology we developed a pre-screening methodology for direct mail pieces. Respondents hold a ten point dial in their hands and they use it to register their responses to multiple choice questions or as a scale from one to ten. We developed a system to test new packages against a control to determine the likelihood of beating it, equaling it or losing to it. Packages are evaluated on potential stopping power of the components, the offer, and graphics. In addition, copy can be evaluated line by line. Furthermore, questions can be added as the research is going on as unexpected issues surface."

Best of both

Through the research, KAET was able to test two mail pieces and determine which was strongest. In the end, based on the research findings, the station has decided to combine the best elements of both packages, says Kelly McCullough, formerly viewer marketing manager with KAET, now with QED Communications, Pittsburgh.

"The two mailers were very similar in terms of graphic presentation and color. There were some different nuances in the copy, but they were very similar packages. When we walked into the test we were thinking, let's see if package A beats package B. When we walked out we said, well, package A wins but let's also integrate the positive aspects of package B."

The four sessions were structured this way: Each began with an introduction section during which respondents used the MACH 2 devices to enter basic information about themselves. This served to train them in using the technology. Next came the actual testing of the direct mail packages. The sessions finished with a focus group to elicit more open-ended responses.

"We added the focus groups for additional depth of diagnostic information, to peel back the onion a little more and allow room for spontaneous reactions that you don't get in quantitative," Schwartz says. "We found that although these were lapsed members, they were very loyal. They felt guilty about their membership lapsing. Some had lost their jobs or had another excuse, but they still wanted to be a part of the station."

Avoid group-think

The focus groups were also helpful, McCullough says, because they allowed the station staff to get individual responses as well as group responses, thus avoiding the group-think syndrome.

"You can get discrete information from respondents first and then get responses from a group within the focus group environment. As you know, in a focus group situation things can often be dominated by one or two people, and I think you really avoid group-think with the MACH 2. I would liken some of the older research technology to Pong, and with this system we've moved into the Nintendo generation."

The respondents viewed the components of the direct mail pieces on slides. This capability gives marketers freedom to test rough executions of ideas before going to production, Schwartz says. "It gives you some leeway because you don't have to go to full production if you want to test some alternatives. It offers a low cost way of measuring them."

Other tests

McCullough says that seeing the system in action in the first session gave the station management ideas about other things to test. "After the first session we started thinking about all the different applications for this methodology, some of which we did not get a chance to use in the four sessions that we had scheduled. Namely, presenting video to the respondents, like pledge breaks, spots, promos, or whatever it might be, to get their feedback."

Two items they did test included program names and the respondents' interest in having the materials "sent" to them by someone associated with PBS. "We decided to ask, who would you like to get a letter from in the PBS world? Is it the local station management, program or membership managers? Or would they rather hear from the so-called PBS personalities. So we threw some names by them and got some very interesting responses."

Hot button

In the line-by-line testing of the cover letter the research found that respondent interest levels went up when the text included the names of specific PBS programs, Schwartz says. "Whenever the letter mentioned specific programming the scores were very high and whenever the letter started with hard sell the scores went down. After the first session program titles seemed to be a hot button. So we added a question to get respondent rankings of 25 programs. In subsequent sessions other issues surfaced, such as funding and station management. As a result we added 60 additional questions. Despite the large number of questions, respondent boredom and fatigue weren't problems."

McCullough: "When the program titles would scroll by the men would respond very differently than women to specific programs. When Great Performances came up the female lines started to soar and the male lines start to fall. When Motorweek came up, the opposite happened. Some of it was predictable but it was quite wonderful to see the responses that the program titles received."

Studies finds more proof that magazine advertising works

Two studies conducted by Family Circle magazine, Citicorp POS and Simmons Market Research Bureau have come up with some pretty convincing evidence of the effects of magazine advertising. The first, the Family Circle Advertising Impact Study, followed the buying habits of nearly 400,000 households in four metro areas during a 28-week period.

The advertising impact study measured the purchasing habits of two groups, one group was exposed to the advertising in an issue of Family Circle, the other was not. (Family Circle has a circulation of 5 million copies, including single copy and subscriptions.) The purchases of both groups were monitored over a 28-week period beginning on December 30, 1991. The period was broken down into three segments, a 12-week pre-period; a 4-week observation period; and a 12-week post-period.

"The most significant finding is that magazine advertising had profound, significant, and immediate effects on sales of the advertised brand. In the past, we weren't credited with that," says Rebecca McPheters, vice president of Information Services at The New York Times Company Women's Magazines, the company that publishes Family Circle.

"In the late 80s and early 90s there has been a lot of talk about advertising accountability and so it seemed that given that the tools were available this was kind of the time to move the ball forward. The other thing that we were able to prove is that magazine advertising does reach people who are disproportionate purchasers of multiple product categories, which is consistent with what we had been hearing from syndicated research for a long time but it was nice to be able to validate that. So you have two effects: the audience effect that accrues from reaching people who are already predisposed toward a particular category and the ad effect which synergistically impacts on that."

Study unique

According to the study reports, the study is unique in five respects because it:

1. Focused solely on the effectiveness of magazine advertising. It did not involve intermedia comparisons.

2. Measured actual behavior--rather than intent--at the household level.

3. Isolated volume effects among households with known exposure to advertising.

4. Did not measure the effects of advertising on either attitude or recall.

5. Did not rely on consumer perceptions of either advertising exposure or of product purchase.

"Most other studies have dealt with changes in awareness, recall, intent to purchase, or perceived purchase and this study clearly wasn't about awareness or intent, it was about actual purchase," McPheters says. "One of the things that we learned was that people's perceptions, for example, of frequency of purchase, are different from actual purchase behavior. When you're asking people what they have purchased within a one month or six week time frame, as many of the syndicated services do, it's difficult for people to remember. . . did they purchase it seven weeks ago or six weeks ago? There's a lot of room for errors in perception."

Four metro areas

The study was conducted with consumers in four metro areas: Richmond, Virginia; Denver; Chicago; and Southern California (Bakersfield, Fresno, Los Angeles, San Diego). Participants were chosen from shopper's clubs of the Von's Pavilion, Ukrop's, Safeway and Jewel supermarket chains.

The sample was generated using data from the Citicorp POS scanner database, which contains information on purchasing habits collected from participating households, who receive program benefits from supermarket retailers in exchange for allowing their purchases to be tracked.

The scanner information identified households that had purchased a single copy of the April 21, 1992 issue of Family Circle. That list was checked against a list of subscribing households to identify subscribers. Of the households exposed to the advertising, 52 percent had purchased the single issue; 48 percent received their copy by subscription.

The non-exposed group was weighted to make it similar to the exposed group in terms of market, household size and income, age of the head of the household, and average number of weekly shopping trips. (While the study controlled for demography, it did not control directly for exposure to other media. However, use of Simmons information on the media consumption habits of Family Circle readers allowed the researchers to make some inferences.)

Increased sales

Nine brands that ran one or more paid four-color ads in the April 21 issue participated in the study. Overall, the study found that magazine advertising increased sales for eight of the nine brands. This was demonstrated by:

- Absolute sales 22.5 percent higher among exposed households during the period of advertising.

- A rate of increase over base period sales 10 percent higher among exposed households.
One of the nine brands, Endust Dusting and Cleaning Spray, experienced a high increase in sales during the test period. In the first half of 1992, Endust advertised only in magazines. In addition to the ad in Family Circle , Endust also ran ads in April in Good Housekeeping and Parade. Two additional Endust ads ran in the January 1992 Family Circle and March 1992 issue of USA Weekend.

During the 12 weeks before the Endust ad ran in Family Circle, the control and exposed groups bought Endust at the same rate. After the ad ran in Family Circle, the exposed group bought 25 percent more. During the post period, those who bought Family Circle continued to buy Endust at a 10 percent higher rate than those in the control group. According to the study report, "There was virtually no change in the Endust purchasing of the control group throughout the 28 week measurement period."

Effect of coupons

The Endust ad featured a coupon, which may account for its effectiveness. McPheters says she suspects that coupons have an impact, but the study didn't measure coupon impact directly. A subsequent study may investigate that.

"I have the strong point of view that coupons make ads more effective. But in fact across the studies we've done we have only measured four ads with coupons. The Endust ad had a 75 cent coupon, which certainly would seem to have generated a high degree of consumer interest but with only four couponed ads measured we can't generalize. Over time as we develop a larger database including more coupon ads we'll be able to address that more directly."

The study also tracked the impact of an ad for Nabisco Teddy Grahams. In the first half of 1992, approximately 65 percent of the advertising budget for Teddy Grahams was spent on television. The remainder of the budget went for purchasing space in six Family Circle and five other magazines, such as Parents and Better Homes & Gardens. During the pre-test period, households that were exposed to the ad in Family Circle purchased 10.6 percent more Teddy Grahams than the households who weren't known to have purchased Family Circle . In the observation period, the control households were outpurchased by exposed households by 21 percent. In the 12 weeks after the ad ran in Family Circle, exposed households bought 20 percent more than the control households.

Both the Teddy Graham and Endust examples also show that advertising had an impact on sales in the products' respective categories. In the ready-to-eat cookie category, the study found a 16.7 percent sales differential between the exposed and control groups during the four-week observation period. Weekly volume per 100 households in the furniture care category was higher for the exposed group than for the control group through all three periods of the study.

Awareness of advertising

The magazine study was followed up with 500 phone interviews with a nationally projectable sample of women to find out if women were aware that magazine advertising affected their purchase habits.

- 58 percent of the women surveyed said that magazine advertising affects their purchase decision.

- Almost two-thirds (64 percent) said that magazine advertising affects their purchase decision even more than TV advertising.

- The respondents said an ad's coupons (86 percent) are most useful, followed by its recipes (81 percent), product information (77 percent), "how-to" tips (73 percent) and "education" (61 percent).

"We had the proof of magazine advertising effectiveness but we wondered if people were aware of it," McPheters says. "It was interesting that in fact they knew that it worked. They were asked about whether they responded to advertising in different media, they said they responded particularly well to magazine advertising."

Schedule impact study

Using a methodology similar to the Advertising Impact study, the second study, the Family Circle Study of Schedule Impact, tracked sales to 80,000 households over 68 weeks. It found that high frequency advertisers achieved significant sales gains that were maintained for the duration of the advertising schedule.

Other findings:

- Advertising schedules generated sustained sales increases for six of the seven brands measured.

- For the six brands, increases in sales attributable to advertising averaged 13 percent.

- The advertising schedules increased sales of the advertised product categories by five percent.

Of the seven brands tracked in the Schedule Impact Study, Spray 'N Wash Stain Stick was a top performer. The brand, which advertised three times in the magazine over a four-month period, saw its sales increase at a rate 37 percent higher among households exposed to its advertising in one or more issues than among the others. For its product category, the rate of increase was 15 percent higher.

"These studies are good news for magazines and for marketers of consumer products," McPheters says. "They provide hard data showing definitively that consumers respond to product advertising in magazines by purchasing more of the advertised brands. We feel we've broken ground in tying magazine advertising to clearly documented changes in consumer behavior.

"In the future, I expect that marketers will have the information necessary to calculate differences in the rates of return generated by various media and promotional alternatives. Manufacturers have a history of using systems like this to track promotions but they haven't used them as extensively to track the effects of advertising and certainly not the effects of advertising in magazines. At least now we know that if you advertise in magazines with enough consistency, you can not only increase your sales but you can also maintain that increase."