Editor’s note: John Weisberg is vice president of marketing at Techneos Systems Inc., a Vancouver, B.C.-based research software firm.

“The MCAPI Primer Part 1: Introducing MCAPI” appeared in the February 2003 issue of Quirk’s. It explained what MCAPI is and how it compares to other data collection methods. This month, Part 2 concludes the primer with advice on how to select and use an MCAPI system.

If you’ve decided to add MCAPI to your marketing research toolkit, the next step is to choose a system that is suitable for your business. Start by analyzing your needs in the following areas, to generate a list of specifications the system must meet to work effectively for you.

  • Project characteristics

Do projects require changes to questionnaire design over time (e.g., longitudinal studies)? Then effective modification tracking is important. What question types are used to get desired information? Must survey designs include randomized response lists, complex skip patterns, and other techniques to ensure data quality? The MCAPI software has to provide the capabilities identified. Do you frequently re-use standard questions and response sets? Software that enables easy re-use of questionnaire elements will be more efficient for you. Will you use very long questionnaires, or require a large number of questionnaires to be on a handheld all at once? The software and hardware must have the necessary capacity.

  • Interviewer and respondent needs

What control would you like the software to have over the interview process? The ability to modify the software user interface can accommodate inexperienced or veteran interviewers, or adapt the system for self-completion. Hiding the software’s menus, modifying “quit” and “save” behavior at the end of an interview, and preventing users from switching to another application on the handheld are some of the key things you may want to control. What unique respondent needs must be addressed? If respondents come from different ethnic groups, support for multi-language questionnaires may be essential.

  • Physical setting

Are interviewers or respondents located at distant or dispersed locations? You will want a system that is easy to support and has reliable data communications. Will the mobile computers be used in extreme environments (hot, cold, dusty, dark)? Hardware suited to these situations is available, but may only work with certain MCAPI software.

These are just examples of the questions to ask. By doing an initial analysis and creating a list of required specifications, you can eliminate any unsuitable software and hardware. Compare additional features of the remaining products to determine which is best for you. The next two sections will help you to evaluate specific software packages and hardware devices.

Software

The advantages of MCAPI over paper - speed, quality, economy and power - were covered in Part 1 of this primer. Compare the extent to which these advantages are provided by competing packages. Check for the key features expected in any computer-assisted interviewing program: branching/skips, range/logic checks, activity logging, etc. Is the system scalable so it can grow with your business?

A system that integrates with other tools simplifies multimodal interviewing. Direct export to your analysis and reporting tools saves time and eliminates errors. Open standards such as the SPSS MR Data Model and Triple-S provide compatibility with the widest range of products.

Mobile computing is attracting developers who know little about survey research or mobile systems. Check the developer’s background, reputation and focus. How available and competent is technical support? Are testimonials and an opportunity to talk to existing users provided? What training, customization and other services are available? Does the developer stand behind its products by offering a free evaluation copy and a satisfaction guarantee?

Popularity of a software package is important. More field houses and MR firms know how to use a popular package, and there is a larger pool of skilled staff to do the work.

The differences between systems must be weighed against their relative price. Total system cost will include software, training, ongoing support and the necessary hardware. A variety of licensing and leasing options may be available. The vendor should help you determine the best return on investment for your particular situation.

Check the CAPI/CASI section of Quirk’s online software directory to find MCAPI vendors, and watch for reviews in future issues of the magazine.

Hardware

The options for MCAPI are primarily the Palm OS and Microsoft Pocket PC. Since its debut in 1996 the Palm OS has consistently outsold all other mobile platforms combined. As a result, Palm OS devices are readily found, at competitive prices, and work with a wide range of available hardware accessories.

The attributes that make a computer truly mobile were covered in Part 1. Generally, Palm OS devices meet the criteria better than Pocket PC devices, especially when it comes to battery life.

Differences between operating systems mean hardware specifications cannot be directly compared across platforms. Palm OS devices with 2MB of memory and a 20MHZ processor can run sophisticated MCAPI software; Pocket PC cannot run on a device with such modest specifications. Try software for both platforms on the type of device you would buy for interviewers to determine what real-life performance is like.

The ability to use less powerful hardware gives the Palm OS a cost advantage. Palm OS devices have the inexpensive segment of the market to themselves with prices less than half that of entry-level Pocket PC models. The savings add up quickly when equipping a field force.

The simplicity of the Palm OS has two added benefits. Its interface requires less training to learn and is less confusing for users. It also tends to be more reliable - system problems are rare on a Palm OS device, and are usually due to poorly written software.

Obsolescence is a concern when buying hardware. So far, Palm’s record of providing upgrade options and backwards compatibility is good, while Microsoft’s is poor. Buying the least expensive hardware that meets your anticipated needs for a year or so avoids the issue - it can pay for itself before it is outdated.

People may share responsibility for the hardware platform decision. Be aware that people may lean towards a platform or model that they would like for personal use or another business application. The requirements analysis outlined earlier helps ensure that the choice will meet your needs, so do not skip it.

The platform decision should include determining that models with the capabilities you need are available. Key items to look for are screen type and resolution; backup and expansion capability; availability of modems, keyboards, and other accessories; and features such as bar code scanning, wireless connectivity, and voice recording. Special features are often model-specific, so check that they are supported by the software you plan to use.

Implementation issues

Anybody who has gone through the introduction of a new data collection method will appreciate that regardless of potential benefits, success is not a foregone conclusion. Pay attention to the following areas when introducing MCAPI.

  • Interviewer training and buy-in

Many interviewers are keen to switch from paper to handhelds and do so quickly, but the ones who don’t can derail implementation. Take the introduction of MCAPI as seriously as you would the introduction of a CATI system. Ensure interviewers receive complete training and good support during the roll-out. Some may not see the benefits that MCAPI provides throughout the survey process, only the disruption it causes in their job, so explain why the change is a positive one for the company and its clients, and how that is good for them.

  • Under- and over-design

Experienced questionnaire designers and scriptwriters may underestimate what an MCAPI system can do and not fully utilize it. Those new to computer-assisted interviewing (CAI) may not even be aware of what could be possible. At the other extreme, watch out for new users of CAI who get caught up in what is possible, and spend time on design that does not result in benefits during the rest of the project.

  • Unrealistic expectations of technology

MCAPI brings the power of CAI right to the field. That is a huge improvement, but don’t expect miracles. Use of a computer does not make open-ended text responses as easy to handle as categorical responses, anymore than it did with CATI. Similarly though, MCAPI offers some advantages over recording them on paper. Be pragmatic and focus on what is possible - you can make substantial improvements with current technology.

  • High rate of mobile hardware technology change

Every technology purchaser faces the dilemma of whether to buy now or wait. If the expected payback period will probably be less than the time until a much better technology is available, then it is worth buying now. Also, it does not matter when a new technology will come out if it cannot generate more revenue or further reduce costs. In most cases, the added benefits of new technology in the near future are small compared to the immediate difference in moving from paper or laptops to handhelds.

Designing for MCAPI

We have covered what MCAPI is, when to use it, how to select a system, and implementation issues. This final section provides tips on how to design questionnaires that deliver the benefits of MCAPI.

Everything you know about good questionnaire design applies when creating MCAPI surveys. However, any notions about design for paper must be put aside. The goal is to get better data - more quickly and with less effort for interviewers and respondents - than is possible with paper. The first rule of good MCAPI design is therefore not to automatically take the same approach you would with paper.

  • Put more time into design. Proper MCAPI design can make interviews faster and easier, reduce errors and data cleaning work and simplify analysis. Achieving these results takes more time at the design stage than when using paper. The effort will pay off in every remaining step in the survey process.
  • Design and present the questionnaire to optimize the interview. Paper questionnaire design optimizes page layout to reduce the number of pages, and make it easy to follow for interviewers, respondents and data entry staff. With MCAPI, the number of “pages” has no effect on cost, navigation is handled by the software, and responses do not need to be re-keyed. Design can focus on making it easy to understand questions and enter responses.

Instead of saving space and simplifying data entry by using a response set with only the most common answers and an “other” category, as you would with paper, use all the items you want to have available for analysis. Capture exact numbers instead of ranges where appropriate. Specific answers are useful for skip patterns and calculations, and can be piped or filtered later in the interview to make it clearer. Break complicated questions down into parts that are easier to display on a handheld-size screen, and are easier for the respondents to understand.

  • Use more complex designs. It may seem counterintuitive to people unfamiliar with computer-assisted interviewing, but complex designs can make the interview easier and faster. Use conditional questions to ensure respondents are asked only those that apply. Complex designs can also reduce errors and obtain more precise answers. Calculations, error checks and recoding can be included without interrupting the interview flow, unless a problem is detected.
  • Take advantage of automated collection. Most MCAPI software will automatically record date, time and duration of interview, respondent number, and language used. Other questions you might ask on paper can be eliminated - for example, expenditures can be automatically totaled. Inferences that would normally be made during data processing to fill in missing responses can be used in real-time to shorten the interview.

Some hardware supports additional types of automated data collection, including bar codes and magnetic stripe readers. Packaged goods product codes, ticket purchase information, coupon codes and more can be accurately and quickly recorded.

Finally, bear in mind two important points about MCAPI design. The length and difficulty of the interview for a particular respondent is the issue, not the total number of questions (screens) in the questionnaire. And good design should free the interviewer to focus complete attention on interaction with the respondent, which is the whole point of face-to-face interviews.

Unique benefits

Research buyers and providers need to be knowledgeable about a wide range of methods in order to do their jobs effectively. MCAPI is an approach with unique benefits. If it suits your work, use this primer to guide you as you give it a try.

Additional resources

PalmSource - Palm OS compatible software and hardware: www.palmsource.com.

Microsoft Pocket PC site: www.microsoft.com/mobile/pocketpc/default.asp.

Pen Computing magazine - cross-platform mobile computing news and reviews: www.pencomputing.com.

SPSS MR Data Model: www.spss.com/spssmr/dimensions/datamodel.htm.

Triple-s: www.triple-s.org.

Check out the sources listed at the end of Part 1 as well.