Editor’s note: John Weisberg is vice president of marketing at Techneos Systems Inc., a Vancouver, B.C.-based research software firm. Part II will appear in an upcoming issue of Quirk’s.
You’re comfortable with the idea of telephone interviewing using CATI. You’re still digesting the impact of Web-based interviewing. But now you’re starting to hear about something called MCAPI. You might be excited about a new interviewing technology that could provide further competitive advantage, or resigned to spending the time to learn about it.
You will find this primer valuable regardless of your reaction. It provides an introduction to MCAPI (mobile computer-assisted personal interviewing), explaining what MCAPI is, how it differs from other computer-assisted interviewing methods and paper-based interviewing, and when to use it. You’ll discover that this interviewing method builds on what you already know and do. It lets you refine existing procedures and focus techniques so that interviews can be conducted more effectively and efficiently, in hopes of reaping greater benefits and satisfaction for respondents, interviewers, and clients. Whether you are an end user or supplier of survey-based research, MCAPI will have an impact on your activities. So relax, and read on to see what’s in it for you.
What is MCAPI?
Computer-assisted interviewing (CAI) has become a well-accepted technology for survey research. There are a number of well-established CAI methods, including CATI (computer-assisted telephone interviewing), Internet-based systems, and CAPI (computer-assisted personal interviewing). As useful as these methods are, however, they cannot reach every population or necessarily reach them at the best time.
If respondents should be contacted at a certain time or place but CATI, Web interviewing, and CAPI are not suitable, what alternatives do researchers have? The choice has been to forgo the benefits of CAI and resort to paper questionnaires, or to contact respondents at a less optimal time or place so that one of the established CAI methods can be utilized. Neither choice provides an ideal data collection solution.
It is possible, however, to expand effective CAI coverage with mobile computing technologies. MCAPI’s addition of mobility provides the benefits of CAI in many situations where pencil-and-paper interviewing was previously the only viable option. It allows researchers to improve current practices, and engage in new forms of research.
This definition of MCAPI is based on existing industry definitions of CAI (ESOMAR 2000):
MCAPI is mobile computer-assisted survey information collection in a face-to-face setting, where an interviewer enters respondent answers directly into a computer, and a specifically designed program manages the administration of the interview. The program controls presentation to the interviewer of the questionnaire elements, and may automatically record certain data, check for invalid responses, and perform calculations, thereby avoiding subsequent editing and keying in of data.
MCASI (mobile computer-assisted self-interviewing) is a variation for self-completion of a questionnaire by respondents. It differs from MCAPI primarily in terms of who enters the data. The survey software user interface varies accordingly. Interviewers can be given initial training to use a more complex interface, in order to gain efficiency on every interview they conduct. For self-completed interviews, the software needs to be usable with minimal initial explanation and no support during the process of completing the questionnaire. Some MCAPI software programs allow enough control over the interface to be used in both situations. Throughout the rest of this primer, MCAPI is used to refer to all forms of mobile CAI.
An MCAPI system typically consists of these components:
- design software for a PC, used to create questionnaires and manage returned data;
- interviewing software for a mobile computer, used to administer questionnaires;
- communications software, to transfer data between the design and interviewing programs (it usually works behind the scenes);
- mobile computing hardware, most commonly a handheld computer.
MCAPI has been around since the late 1980s, so you might be wondering why, if it is useful, it isn’t more common. One reason is that until the release of the Palm Pilot handheld computer in 1996, mobile computers weren’t successful in the marketplace. Then the Internet tidal wave hit. Researcher buyers and suppliers, and most software developers, have been too busy coping with the impact of the Internet to consider mobile solutions. People are finding the time to look at other technologies, though, and the ability to access data anytime, anywhere by combining Internet and mobile technologies is driving most development work currently underway. MCAPI and mobile computing will inevitably become part of the researcher’s toolkit over the next few years.
What makes it mobile?
“Mobile” indicates a critical distinction from CAPI. The low mobility of personal computers has restricted their use to certain locations. Face-to-face interviewing (i.e., CAPI) is often defined in terms of mall intercepts and door-to-door surveys. Central location interviewing (e.g., a mall facility) requires that respondents naturally be at that location at some point in time. If not, participation in a survey requires additional time and effort on their part. On the other hand, hauling a laptop computer to the respondent’s location is an added burden for the interviewer. Mobile computers make it much easier to go to where respondents are, wherever that may be, in order to collect data.
A mobile computer is one designed to be used while moving about. This means the research design no longer has to fit the places where other CAI methods can be used. Instead, a researcher can choose the data collection method and location that best meets criteria such as minimizing respondent burden, increasing response rate, limiting cost, and reducing overall error.
Experience and studies have shown that hardware with the following attributes is required to achieve the degree of mobility that makes MCAPI an effective alternative to other CAI methods:
- Low weight is essential when using a computer while moving about. It also eases transport or shipping to other project locations. Weight below one pound (450 grams) is preferable.
- Long battery life is required to be productive in the field. A minimum of eight hours run-time is recommended. Ideally, batteries should be user-replaceable, as most hardware problems in the field are caused by batteries. At a minimum, built-in batteries should recharge rapidly, and be rechargeable from additional power sources besides AC line current.
- Ergonomic design for mobile use. This includes size and shape, to make it easy to hold and use. It also covers interface design and input method, so that the interview can be easily navigated and data entered efficiently. A pen-based interface is superior to a keyboard when using the computer while standing or moving about. A keyboard can supplement pen input for open-ended questions.
- Motion-proof hardware so that it will not be damaged when using the device on the move.
- Instant-on capability so the interview software can be accessed immediately without waiting for the computer to start up.
Desktop and laptop PCs do not have these attributes, and are unsuitable for mobile interviewing. Handheld computers are currently the most appropriate choice, but MCAPI is not defined by the use of a handheld device. Other formats such as wearable computers are being developed. As different devices become more common and affordable in the future, they will provide new capabilities for mobile interviewing and observational studies. MCASI will continue to require compact devices that can be easily handed to a respondent.
Why use MCAPI?
The primary reason to use MCAPI is to effectively provide the benefits of CAI in more situations. It can replace paper directly, or supplant another CAI method that was used to replace paper when no better alternative was available.
There are four main advantages CAI methods such as MCAPI have compared to paper: speed, quality, economy, and power.
Speed means reducing the time it takes for each stage of a research project, as well as the total length. It is an advantage because research users need results faster than ever. Saving time also saves money, by improving productivity or making it possible to reach a decision sooner. It can also have a positive impact on quality - if initial interviews can be analyzed in hours instead of days or weeks, problems can be detected and resolved soon enough to improve subsequent interviews.
Quality means preventing errors, dealing effectively with errors that occur, and collecting more exact information. The benefits include more reliable results, reduced respondent and interviewer burden, and lowered costs.
Economy means providing at least as much speed, quality, and design power as paper, but at a lower cost. That can improve a firm’s profitability, or allow more research to be completed with the same budget.
Design power means creating a questionnaire with built-in “intelligence” that optimizes its performance given the research objectives, respondent and interviewer characteristics, and interviewing situation. Paper questionnaires have no brains; the interviewer or respondent is entirely responsible for any operations that are required to complete the interview, such as skips, calculations, limiting the number of responses, etc. A good CAI program can automate such operations, and allow the designer to create a questionnaire that asks more precise questions without placing greater demands on the interviewer or respondent. The power of CAI also enables research approaches that simply are not possible with paper.
MCAPI in action: Example 1
The Liquor Control Board of Ontario (LCBO) is the world’s largest purchaser of beverage alcohol. Needing to assess customer response to its newest flagship store in Toronto, the LCBO faced a problem: how to apply its previously developed customer segmentation model in real-time during in-store interviews, so appropriate in-depth follow-up could immediately be conducted with members of each group.
MCAPI provided the solution. Automatic calculations evaluated responses to a set of attitudinal questions against established scores, and immediately indicated the group to which each respondent belonged.
Bob Collins, director of the LCBO’s Customer Insights Group, feels the project would have been virtually impossible to do with PCs or laptop computers, and the calculations involved meant paper was out of the question. The questionnaire was much easier for interviewers to administer, so they were able to complete more interviews. Customers found the method more interesting than pencil-and-paper, resulting in a higher rate of cooperation. And unlike other survey formats, he says, there was no need to re-key the collected data.
Replacing paper
Greater speed, quality, economy, and power benefit research users and suppliers, interviewers, and respondents. While paper may still be the most commonly used method for collecting survey information, the advantages of CAI methods mean they are replacing paper whenever it is practical and cost-effective to do so. MCAPI offers the advantages of other CAI methods but also challenges the two qualities that have maintained the value of paper as a data collection technology: mobility and low initial cost.
Paper is superbly mobile. It can be mailed to individuals, shipped en masse, or carried in person to the required location. Its use for interviewing does not require any infrastructure: no phone lines, Internet connections, or electrical plug-ins required. Since it is such a widespread technology, there is minimal investment to use paper - researchers already have the equipment and skills to work with paper questionnaires - and respondents are comfortable using it themselves.
Handheld computers have equally impressive mobility, with somewhat different characteristics. Their low cost means that although an up-front investment is required, it can be recouped relatively quickly by eliminating the incremental costs of using paper. MCAPI can’t replace paper in all situations, but there are many times when it is a better choice.
Table 1 shows the advantages of MCAPI relative to paper questionnaires for various activities in a research project.
 |
While MCAPI provides some time savings in the design stage (e.g., re-use of questions and response lists), it is normal to spend more time on design than with paper. Additional work at this stage to build intelligence into the questionnaire pays off in time and cost savings, and improved quality, at later stages. In addition, the ability to quickly modify the questionnaire and distribute new versions, without needing to print and ship paper forms (and throw away those already printed), encourages taking the time to refine the questionnaire. This results in faster and more trouble-free analysis. Overall, spending more time on MCAPI design is worthwhile.
MCAPI costs
MCAPI has a much different cost structure than PAPI. It requires an up-front investment in software, hardware, and training, but eliminates most incremental costs except for design and interviewing time. Paper requires a minimal investment but has many incremental costs, on either a project or interview basis. Table 2 compares costs for the two methods.
 |
These differences mean the two methods are cost-effective under different conditions. MCAPI is most cost-effective for projects with a relatively small number of interviewers, and a large number of completed interviews and/or a long and complex interview. A low number of interviewers requires less initial investment, and a large number of interviews, or a long questionnaire, means significant incremental costs to use paper are avoided. The more projects and interviews an MCAPI system is used on, the better the return on investment will be.
On the other hand, if on-site data collection is required infrequently, and each interviewer will conduct only a few short interviews, the cost savings provided by MCAPI may be insufficient to cover the investment required within a reasonable time.
The relationship between total cost and the number of projects and interviews is shown in Figure 1. MCAPI has a higher initial cost, but at work volumes to the right of where the two cost lines cross, it is more cost-effective than paper.
 |
You can roughly calculate return on investment for MCAPI by comparing the initial cost of the MCAPI system (software, hardware, and training) with the money saved by eliminating questionnaire printing and shipping, data entry setup, and data entry. MCAPI software vendors may have more detailed ROI calculators available.
Additional MCAPI advantages
CATI and Web interviewing have been studied to determine such things as respondent comfort and satisfaction with the method. MCAPI has not yet been examined to the same extent. The research that has been published, and comments by users, do indicate that the method is viewed positively by respondents, interviewers, and clients.
Respondents generally view interviewers and the organization sponsoring the research as more professional when handheld computers are used instead of paper and a clipboard. This is especially true for complex interviews where MCAPI avoids the delays and confusion of figuring out which question to ask next. Interviewing outdoors in windy conditions does not require contortions to keep the paper flat on the clipboard, or mad dashes to capture questionnaires that have blown away. There is evidence from other CAI methods that respondents feel their answers are more accurately recorded, and more confidential, when entered into a computer.
Interviewers feel more professional when using MCAPI. In part this comes from eliminating some of paper’s problems, such as fumbling with multiple sheets of paper, and trying to figure out what to ask next while the respondent waits. Interviewer self-worth seems enhanced by being trusted with a piece of sophisticated equipment. The benefits are that interviewers have a more professional attitude towards respondents, and the ability to focus on the interaction with them instead of on handling paper and navigating the interview.
Research buyers appreciate the professionalism and quality that MCAPI provides compared to paper. This is not surprising given that they have come to expect the advantages of CAI for other types of data collection, such as telephone interviews.
MCAPI in action: Example 2
FairControl is a company that provides business intelligence services to major exhibitors at trade fairs, conferences, seminars, and other events around the world. Bjoern Jopen, founder and co-chairman, sees a real impact from his firm’s move to MCAPI. “The business has grown. If we approach a client, especially in the IT or telecoms industry, they can instantly see we are professional and innovative. It solves many of the logistical problems of interviewing and means we can be much more flexible. We can react very fast to changes. It also saves a lot of time and stress for our project managers. This clearly gives a better impression to the visitors and exhibitors to trade fairs where we interview.”
When to use MCAPI
MCAPI instead of paper
Paper is an exceptionally mobile format, but there are times when MCAPI should be used instead. Certain characteristics of the project and research design indicate when MCAPI is preferable.
- Speed is vital
Paper questionnaires must be laid out, printed and shipped. Once the questionnaires are printed, any modifications add cost and delay the project. It takes less time to design and deploy a questionnaire with MCAPI (design may take longer, but the speed of deployment more than makes up for it). The press of a button can download or transmit a questionnaire to handhelds around the world, and interviewing can begin at once. A good MCAPI system allows quick modifications, even after going live, while tracking all changes to ensure data integrity.
Retrieving MCAPI data is just as rapid. The data contain fewer errors, so cleaning takes less time. Some MCAPI systems will effortlessly transfer the data into analysis and reporting tools, saving even more time. Paper must be physically returned, and the responses keypunched – a source of delay and additional errors.
- The design is complex
On complex surveys, MCAPI simplifies life for the interviewer by automatically managing the flow and checking for errors. The questionnaire designer can ensure each respondent gets the right questions without being concerned about whether the interviewers can figure out which are the right questions. Automatic validation of data in the field can be built in, using range and logic checks, so inconsistent answers are discovered when they can still be corrected by asking the respondent for clarification. Calculations can be performed invisibly by the system. MCAPI also eliminates the possibility of higher error rates when entering data for complex questionnaires from paper.
The quality of the interview is improved for respondent and interviewer. The interview is faster because there are no sheets of paper to sort through, and no getting lost in a complex design. There is no risk, with proper design, of confusing or offending a respondent by asking inappropriate questions.
- Quality is paramount
Quality may be more important than the cost of data collection when errors have significant costs. Drawing inaccurate conclusions about a new product introduction is one example. Another is ruining a unique data collection opportunity, such as an exit poll during an election. Quality of the interview experience may be a concern with certain groups that are hard to involve in surveys, such as doctors. Even when the other advantages of MCAPI cannot be realized on a project, improved quality may make it the best method.
MCAPI in action: Example 3
Aaron Percefull from GLS Research in California explains difference MCAPI makes for one of the firm’s clients. “Our client is particularly happy with the handheld interviewing technology because it permits them rapid access to data. Since September 11, the client has asked us to conduct year-to-year comparisons by month of Las Vegas visitor behavior. We are able to produce these reports within a very short time after data collection, whereas if the surveys were still being done on paper, we would need at least a couple of weeks just for keypunch. So, the big advantage of [MCAPI] is the elimination of time-consuming, costly, and error-prone keypunch of pencil-and-paper interviews. Because [MCAPI] permits a very high level of internal error checking and verification through the use of scripts, interviewer input errors can be kept to a minimum.”
Paper instead of MCAPI
There is one case when paper may be superior to MCAPI due to its low initial cost and widespread use. This is when many people must be surveyed at one location in a very short time frame. Imagine you need 800 completes from people who are leaving a special event. You have 20 minutes before everyone has left the site, and the questionnaire takes five minutes. A large number of interviewers and handhelds (267) would be required for the project using MCAPI. Because most people know how to fill out a paper questionnaire with minimal assistance, a self-completed paper questionnaire could be distributed and collected by a much smaller number of field staff, and no handhelds would be required. The cost would be much lower for paper, making it the right choice if it can handle the design and provide adequate speed and quality.
MCAPI instead of other CAI methods
Replacing paper is the role MCAPI fulfills best. However, you may already be using another CAI method instead of paper in situations where MCAPI is more appropriate. Certain characteristics of the setting and respondents indicate when MCAPI interviews on-site will be easier and produce better results than other CAI methods.
Setting and experience matter
Respondents’ views can be greatly influenced by the setting they are in. They are also far more willing to talk about an experience when it is relevant to them - typically in the midst of it or immediately afterwards. Responses from people who are commenting directly on the experience may be very different than those based on their intentions before the experience, or their recollections afterwards. Finally, an interviewer on-site can observe the respondent’s actions within the environment, in order to validate responses or gain additional insight. When the setting matters or the experience is transitory, but respondents cannot be reached at the right time or place with phone or PC-based Web interviews, then going on-site with MCAPI is a desirable approach.
Retail, health care, and tourism are some of the areas where setting and immediate experience have proven to be extremely important. In addition, there are specific on-site research techniques such as “assessment in context,” “in-situ research,” and “experience sampling.” MCAPI delivers the advantages of CAI in all these situations.
MCAPI in action: Example 4
Portland, Ore.-based Sorensen Associates conducts in-store research. According to Sorensen’s Michael Gibney, relevancy is a key to getting good data on purchase decisions. “Ask about what they are buying when they are buying it, not when their mind is on something else. People are more willing to participate. You get better answers the closer you get to the checkout - to the point of actually laying down money for a product.” Sorensen’s clients are often consumer packaged goods manufacturers working on new product development, and the speed of MCAPI is very valuable to them. “We can cut two to three days off a project, from design to top-line, compared to using paper,” Gibney says. “MCAPI takes the questionnaire logic away from interviewers and puts it into the handheld device, so less training is required, and errors are reduced.”
Respondents are dispersed or low-incidence
Need to reach respondents who are widely dispersed? If you can identify a time and location where they gather, then interviewing at that point is a very efficient approach. For example, an amusement park may draw visitors from all over the world. Reaching them on-site is easy; contacting them any other time is a logistical nightmare.
What if respondents have a low-incidence in the general population? Normally it is expensive and time-consuming to get an adequate sample. If they are drawn to a particular location or event, however, a concentration of respondents with the particular characteristics you seek may be created. Interview at special or recurring events, such as arts festivals or medical conventions, or at locations that cater to the group, such as woodworking stores or daycare centers.
The key to efficient surveying of widely dispersed or low-incidence respondents is to identify occasions where they gather, and meet them there. If mobility is required to do so, then MCAPI is the most appropriate method.
MCAPI with other methods
Specific project considerations can lead to the choice of multi-mode data collection. In such cases, tying together data from different modes is always a concern. MCAPI is capable of doing this in a number of ways, from systems that offer a common design tool for all CAI methods including MCAPI, to linking based on automated date and time stamps, to the ability to scan data using bar codes or magnetic stripe readers.
MCAPI in action: Example 5
Parks Canada is the agency is responsible for Canada’s National Parks and National Historic Sites. The agency needed information about tour bus groups, which make up a significant proportion of visitors to locations in the Yukon Territory. Two key problems the data collection method(s) needed to address were:
1. Getting enough data. Buses stopped at a location for about 20-30 minutes, making it impossible to personally interview more than one or two of the 40 or more passengers on each bus;
2. Getting reliable data. “If it’s Tuesday this must be...?” The intensive schedule of the tour groups meant passengers were not reliable sources of information about where they had come from and were going to next.
The solution was to combine paper questionnaires and MCAPI interviews. The bus drivers had nothing to do during the stops, so they were interviewed using MCAPI while the passengers visited the site. The interview collected data about the tour bus company and the group’s itinerary, from someone who could provide accurate information. As the passengers returned to the bus, each was handed a mail-back OMR (optical mark reading) scannable questionnaire. Unique bar-code serial numbers pre-printed on the questionnaires were scanned into the interview record on the handheld as the questionnaires were given out. The passengers completed the questionnaires on the way to the next stop, where they could simply be dropped into a mailbox. Passengers had fewer questions to answer because information about the group had already been provided by the driver. When the returned questionnaires were processed, the OMR scanner read the bar-code serial numbers again, providing a highly accurate link to the MCAPI interview data.
In this case, the research user and fieldwork suppliers knew what data collection methods were available, and the strengths and limitations of each. By thoughtfully combining different methods, a way was found to collect enough data and ensure high quality, while making the survey as effortless and enjoyable as possible for respondents and interviewers. That’s a strong argument for becoming familiar with different methods.
Choosing MCAPI
So now that you have a broader selection of interviewing methods available, how do you choose among them?
Depending on the research question, one method may be more appropriate than another. Part of the expertise you must develop as a knowledgeable client, researcher, or fieldwork supplier is knowing which method is most suitable for a particular interviewing situation. At times projects may even require multiple methods.
A tourism research project, for example, may look at all the phases of the tourism experience: anticipation, travel to destination, on-site, return, and recollection. A pop-up Web survey on a destination’s Internet site would be one way to ask questions about the anticipation phase, but MCAPI would not be very useful. On the other hand, phone or Web interviewing is not feasible during the travel, on-site, and return stages. Picking the right method(s) for a particular stage will get the desired information most efficiently. The only way to make that decision correctly is to know the strengths and weaknesses of each method.
 |
Table 3 summarizes the suitability of MCAPI compared to other CAI methods and paper.
Not always the answer
As with all new technologies and methods, MCAPI promises benefits, but quite literally is not the answer to every data collection requirement. This first part of this MCAPI primer explained what MCAPI is and when it is appropriate. For those of you who already see a role for MCAPI in your research activities, Part 2 will discuss the parameters for choosing a MCAPI system, and the implementation issues you should anticipate. Those who aren’t yet sure how they could benefit from MCAPI but want to learn more should check out the resources listed below.
Suggested resources
Techneos Systems Inc. Mobile data collection case studies, bibliography, and newsletter; hardware selection information (www.techneos.com).
Point-Of-Purchase Study Group (POPSG). Contacts and bibliography (www.popsg.org/index.html).
Tim Macer Services. Software directory and reviews (www.macer.co.uk).
Association for Survey Computing (ASC). Software directory, conferences, presentations about different CAI methods (www.asc.org.uk).
Couper, M.P., R.P. Baker, J. Bethlehem, et al. (eds.) (1998) Computer Assisted Survey Information Collection. New York: Wiley. The only book about CAI, it does not cover MCAPI in depth, but identifies relevant issues for all forms of CAI.
Stewart, William P. & R. Bruce Hull (1996) “Capturing the Moments: Concerns of In Situ Leisure Research.” In Daniel R. Fesenmaier, Joseph T. O’Leary, & Muzaffer Uysal (eds.) Recent Advances in Tourism Marketing Research. New York: Haworth Press: 3-20. Co-published in Journal of Travel & Tourism Marketing 5(1/2): 3-20.