Editor’s note: Allen Hogg is director of marketing for Burke Interactive, the Internet research support group at Burke, Inc., Cincinnati.

Multi-mode research is appropriate not only when companies like Fidelity want to give customers a choice of how to respond to surveys. It can also be a method for reducing survey coverage and non-response error in many other situations. (Coverage error occurs when individuals invited to participate in a survey are not representative of the target population. Non-response error occurs when individuals responding to a survey invitation differ from the entirety of those sampled.)

What is here being called multi-mode research should not to be confused with, for example, a more traditional phone-mail-phone approach, in which respondents are recruited by telephone, sent some materials to examine, then called back to get their responses to questions. Instead, multi-mode research here refers to using two or more data collection methods for the same study wave. Appropriate multi-mode applications include the following:

  • When e-mail addresses are available for only part of the target population (a company’s customer base being the most obvious example), e-mail invitations can be used to invite that segment to take an online survey, while more traditional recruitment and data collection methods are used for other segments of the population.
  • When e-mail addresses are available for an entire population group, telephone “backfill” interviewing can be used to try to get responses from those who did not react to the initial e-mail invitation.

Providing a Web survey option as well as a telephone option can be particularly appealing when conducting customer satisfaction studies in business-to-business situations. Frequently, a telephone-only approach to business-to-business customer satisfaction will require calling potential respondents so many times that the survey process itself has the potential to make the customer dissatisfied. It also has the potential to over-emphasize the views of customers who prefer a “high-touch” approach - and want lots of contact from sales representatives, for example - over more “high-tech” customers who would appreciate a more hands-off business relationship.

Multi-mode approaches can also prove beneficial for global research projects. In these situations, different data collection methods might be used as the “lead mode” in different countries. This has long been done with telephone and in-person interviewing — and now online options can be considered as well. Burke experience with multi-mode international projects has shown that, even when e-mail addresses are available for a global business-to-business customer base, the percentage of respondents using the Web will tend to be far greater in the United States and Canada than in any other area of the world.

Multi-mode projects can certainly demand more administration time than projects utilizing just one data collection mode. Companies may need to put together a Web survey program as well as a CATI script or paper survey instrument. There is also a need to be vigilant about sample management to avoid, for instance, calling respondents after they have completed Web surveys.

Questionnaires should also be designed with the aim of avoiding measurement error that might arise from differences in response patterns due to survey method. This is particularly a concern when a self-administered method, such as Web surveying, is combined with an interviewer-administered approach, such as telephone surveying. The goal of researchers should be to create questionnaires from which findings can be combined without any need for recalibrating results from one data collection method to make them comparable to the other. (There may still be differences in numbers, but they should be due to actual differences in the populations responding via various survey modes, not due to the survey instruments themselves.)

The Fidelity experience reinforces Burke’s finding that telephone interviewing will result in more use of scale end points when semantic rating scales are employed. When there is not a historical data record to consider, using numerical scales with anchored end points leads to much more comparable findings from phone and Web surveys.

Inclusion of “don’t know” or “no response” options is also a key consideration. When such an option is included in a Web survey, the percentage of people using it will tend to be higher than it is for telephone surveys, particularly on sensitive issues. This makes sense when one considers that in telephone research, “don’t know” responses are typically accepted by interviewers, but not volunteered as a possible option. On the other hand, omitting “don’t know” or “no response” options has not been shown to increase Web survey drop-out rates. “Forcing” people to respond does not seem to cause them to exit the survey.

Researchers do not, however, want to force people to respond when they might have no legitimate basis for an opinion. Burke’s default approach has therefore been to leave out “don’t know” and “no response” options from Web surveys on items that will be key to the analysis of the study (with the exception of very sensitive items, such as household income), but to include them with, for example, attribute ratings, when the question might address some aspect of a product or service with which the respondent has no experience.

“Check all that apply” items are another area of concern. Typically, for telephone surveys, such items are structured as a series of “yes” and “no” responses. If not set up the same way for Web surveys, respondents will have a tendency to check only a few items, then move on without considering all the possibilities on the list.

Finally, it is also a good idea to avoid most of the whiz-bang capabilities of online surveying. Although the Web allows survey designers to use things like sliders and visual scale images instead of more traditional rating scales, such devices make it less likely that findings from online and other survey modes will be comparable. Simpler, familiar Web interfaces will serve companies better as they try to use multi-mode research to reduce non-response and coverage error without introducing measurement error into the process.