Half-empty or half-full?

Editor’s note: Jane Sheppard is director, respondent cooperation for the Council for Marketing and Opinion Research, Port Jefferson, N.Y.

The Council for Marketing and Opinion Research (CMOR) was formed in 1992 by the four major associations in the marketing and opinion research field: American Marketing Association (AMA), Advertising Research Foundation (ARF), Council of American Survey Research Organizations (CASRO), and Marketing Research Association (MRA). These associations have been joined by many corporations and organizations that conduct and/or use survey research. Together under the CMOR umbrella, these organizations are working to further the acceptance of survey research by the public and the government through education, lobbying, and providing legislative support to prevent abuses of the research process. CMOR also supports programs that ensure access to consumers, so that respondent cooperation in research remains vibrant and healthy.

To address this specific initiative, a Respondent Cooperation Committee was formed to achieve these goals: to evaluate the public’s perceptions of the research process, to measure the effects of alternative methods of improving respondent cooperation, and to provide a foundation upon which to build an improved set of industry guidelines. It is the responsibility of the CMOR Respondent Cooperation Committee to oversee the organization’s initiatives. Specifically, the mission of this committee is to lead the research industry to improve respondent cooperation by providing: 1) objective information about cooperation rates, consumer understanding of and concerns about research, and the ever-changing research environment, and 2) industry-accepted and supported solutions to improve respondent’s willingness to cooperate.

As a step to achieving those goals, in 1999 CMOR conducted the Respondent Cooperation & Industry Image Study for the second time, using the same questionnaire as in 1995 with some minor revisions/updates. The study was an outgrowth of previous surveys on the public image of the survey research industry conducted by Walker Research and therefore the findings are reflective of trends over time. The primary objectives of the study are to 1) serve as an industry benchmark for future measurement of cooperation levels, and to 2) generate data that might be used to formulate guidelines on best practices.

Methodology

A total of 1,001 computer-assisted telephone interviews were conducted among a random sample of adults nationwide from September 11 through October 12, 1999. Eleven research companies conducted the interviews (at no cost to CMOR): Coast-to-Coast Research, Elrick & Lavidge, FRC Research Corp., Maritz Marketing Research, Market Directions, Inc., NETWORK (Irwin Research, Luth Research, JRA, McMillion Research, Pat Henry Research), The NPD Group, Quick Test, Inc., Roper Starch Worldwide Inc., The Wats Room, and Western WATS Opinion Research Center. Affordable Samples, Inc., provided the RDD samples for each of these companies to use, also at no cost to CMOR. And, NETWORK provided the questionnaire programming, while Computers for Marketing Corporation tabulated the data also at no cost. The interview averaged 19.7 minutes in length.

Conclusions

Impact of the Internet - While not even on the radar screen during the 1995 CMOR survey, Internet surveys have shown substantial growth in the past few years. Nearly one-in-five individuals who have participated in a survey in the past year indicated that their last survey was conducted via the Internet. This approach is proving to be an acceptable (and in many instances desirable) alternative to other survey methods. The methodology currently most impacted by Internet surveys is mail. However, as the Internet achieves greater and greater household penetration, it should be expected that all methods will lose at least some share to it. On the positive side, however, is the evidence that Internet surveys can expand the use of survey research, and therefore the size of the market overall.

Internet focus groups have yet to achieve noticeable penetration. Most likely, the available technology limits the usefulness of this technique. However, as advances occur, this methodology may be used more widely and could be a significant threat to the current central location focus group methodology.

Threats to telephone surveys - A variety of factors are making it increasingly difficult to reach consumers for telephone surveys. As a result, the respondents who complete surveys don’t constitute a random sample of the population of interest to the researcher.

The following factors also increase the cost of conducting telephone surveys:

  • Answering machine ownership is high, and while most owners only screen a relatively small percent of their calls, the overall impact is that much more effort needs to be expended to reach these individuals.
  • Call screening services, especially Caller ID, are gaining in acceptance and penetration.
  • Telemarketing activity is so strong that consumers are feeling hounded. Attitudes towards telemarketing and receptiveness to these calls is declining. And, the level of sugging (selling under the guise of research) activity on the part of telemarketers continues unabated. This has led to an overall decline in attitudes toward unexpected calls in general, which dampens willingness to participate in surveys.
  • There are some individuals who go particularly out of their way to make themselves inaccessible by having unlisted numbers and utilizing a variety of call screening methods.

Personal privacy concerns - There is a high awareness of privacy issues among the general public and a high level of concern for personal privacy. Industry efforts and rules in this area are not being clearly communicated to and/or understood by the public.

Many participants do not recall being told that their answers will be treated confidentially and/or that no one will try to sell them anything as a result of participation. Whether they recall being told or not, a majority do not believe that the marketing research industry protects confidentiality and/or has privacy protection incorporated into the methods.

Attitudes and behaviors toward the survey research industry - Attitudes toward the marketing research industry, while still high on many issues, are showing a slow erosion over time. It appears that much of this can be attributed to the industry’s decline in consideration for survey participants as well as a decline in standards.

Survey lengths increased to 14 minutes in 1999 from eight, 10, 12, and 13 minutes from 1988 through 1995, respectively.

The use of incentives is down considerably. When incentives are used, they are used primarily for focus groups, long interviews (over 20 minutes), and Internet surveys.

Participants perceive a decline in the professionalism and courtesy of interviewers and a decline in the professional appearance of self-administered surveys.

There is an increase in complaints among mail survey participants about not being given enough time to respond, and among telephone and in-person participants about being contacted at inconvenient times.

Political and issue polls appear to be even less well regarded than commercial surveys.

Refusal rates - The refusal rate continues a slow but steady upward climb. In most instances, refusals are circumstantial (i.e., inconvenient time, uninteresting or inappropriate topic) rather than a general reluctance to participate. Most refusals occur before the survey introduction. Very few terminate the survey once it’s underway.

The 1999 survey included a statement in its introduction that a call may be monitored for quality purposes. By law in some states, research firms are now required to include this statement for RDD studies. The data do not show any detrimental effect from the inclusion of this statement.

While most individuals express willingness to participate in future surveys, this willingness is lukewarm and down from 1995. What is most likely being communicated is that actual behavior will be dependent on the circumstances.