From telephone to the Web

By Bill Stone | May 1, 2001

Reading time: 4 minutes

Abstract

While the Internet is not a research utopia, there are often cases where it is a methodologically better mode than the telephone. This article discusses one research team’s success using e-interviewing, including respondent group requirements, fielding requirements, transition to the Web and data quality.

Research Topics:: Customer Satisfaction Studies | Online Research | Quantitative Research | Telephone Interviewing/CATI
Content Type: Research Methodology

Share Print

Editor’s note: Bill Stone is president of Phoenix, Ariz.-based WKS:CSM Consulting, Inc.

The Internet is not a research utopia and its limitations should not be overlooked when seeking to convert from more traditional forms of research. However, there are often cases where it is a methodologically better mode than the telephone.

Our firm has been providing customer satisfaction measurement and problem resolution services to a large international industrial manufacturing and service corporation. For the past five years, we have conducted quarterly telephone surveys using CATI software. This medium is usually effective in collecting survey data. However, we found that methodological shortcomings failed to bring our client maximum value from this study. The shortcomings are due primarily to the requirements of the respondent group and the study’s fielding requirements.

Respondent group requirements

The respondents in this particular sample travel frequently and are usually away from their office location and their work telephone number. Because of the constant travel, evening/weekend interviewing is not possible with this audience. The field service must place an average of 16 calls before actually reaching each of these respondents. Not only do these additional attempts significantly add to the field time and cost of the study, they also detract from the overall response rate. When the respondents are at last contacted during the work day, gaining participation is extremely challenging due to their hectic and busy schedules. This negatively impacts data quality.

Because of their mobility, this respondent group relies heavily on e-mail and Web-based information. They are extremely comfortable using the Internet and prefer e-mail over the telephone as a primary method of communication. Consequently, interviewers encounter even more resistance when attempting a telephone survey.

Fielding requirements

In this study, it was clear that a 24/7 interviewing schedule would provide more accurate data and better customer service. However, due to cost and time restrictions, telephone data collection can only be done on a periodic basis. Additionally, because the data collected via telephone interviewing takes time to process, the client is hampered in its efforts to provide its goal of a 24-hour turnaround of information for the problem resolution process. Problem resolution is one of the primary reasons for making the calls to begin with.

Because the client has an international presence, properly representative sampling requires quotas which demand a large number of interviewers fluent in an array of languages. This is a staffing challenge for even the most adept telephone center. In addition, interviewer dialect differences and accents can lead to a respondent misunderstanding the questionnaire, resulting in inaccurate data.

Transition to the Web

In an effort to remedy these methodological shortcomings and provide maximum value to our client, we began to investigate other modes of market research. Since the respondent base is extremely Web-savvy, an online survey seemed the most feasible solution. However, in making this transition we were forced to address concerns such as sample control, data validity, and detailed probing. These factors are policed in a telephone survey environment, but cannot be controlled in a typical Web environment.

The article “E-interviewers add human touch to Web-based research” in the July/August 2000 issue of Quirk’s (accessible via the online Article Archive at www.quirks.com) seemed to indicate that interviewer-assisted, Web-administered research would answer our methodological concerns while allowing us the advantages of Web-based research. SurveyGuardian, a service of humanvoice, a Provo, Utah, data collection company, does exactly this, combining live interviewers with online surveys. Intensive testing of this service led to our decision to shift the majority of the research to these online programs. Since the transition during the third quarter of 2000, we have been able to produce superior data with a higher response rate and a faster turnaround time, while cutting our field costs by up to 20 percent.

Data quality

Respondent accountability and data accuracy are assured through active monitoring of each survey by e-interviewers. Using the interactive chat feature within SurveyGuardian, interviewers check the respondents’ answers for accuracy and probe open-ends for more clarity as the survey is completed. Since the software allows one person to monitor multiple surveys at once, we have been able to increase our quotas and service more customers, while keeping the same interviewing staff. Direct comparisons of data previously collected via CATI with the new Web data allow us to control for mode affects.

Non-response bias and sample databases are controlled through use of mixed mode contacts. Both e-interviewer and CATI follow-ups are used for non-responders. This multiple media approach can be efficiently handled by the same e-interviewing staff, thus maintaining interviewer familiarity and avoiding added training and staffing costs or methodological differences induced by different interviewers.

The Web-based approach has also allowed for delivery in multiple languages across vast time differences. The multi-language process has been simplified and the text-only interaction between the interviewer and the respondent eliminates any differences in dialect.

Easier to reach

Study invitations are issued via an e-mail message containing a direct link to the survey. Since the respondent base has access to e-mail in any location, it is easier to reach the predefined target audience. Respondents are able to complete the survey at their convenience. Consequently, we have seen a dramatic rise in response rates. Since the cost savings allow for year-round interviewing, we project even better respondent cooperation rates in the future.

We are able to access our data in real-time resulting in fast attention to respondent concerns. The software also allows us to track respondent rating histories to determine if previously reported problems have been resolved. This allows our client to deliver superior customer service.

Web-based research alone cannot overcome the shortcomings commonly encountered with self-administered questionnaires. However, in this study we found that, when combined with emerging technologies like e-interviewing, the Internet can be a viable tool for meeting a wide variety of market research needs.