Do it yourself, Web-style
Editor's note: Tim Glowa is a president of North Country Research, a Calgary, Alberta marketing science company.
Self-completed Web surveys offer many advantages, including lower data collection costs, shorter data collection time periods, enhanced survey controls (namely automated skip patterns or forcing responses when applicable), and reduced data entry errors. But the enormous potential of Web surveys must be balanced against an equally large weakness. Most North American households do not have computers or Internet access. In October 1998, only 42 percent of U.S. households reported owning a computer, up from 24 percent in 1994. Another study, conducted by CBC/New York Times in 1998 suggested that two-thirds of all households contacted have access to a computer, and a third of these respondents have Internet access (Dillman 2000, Mail and Internet Surveys). While the proportion of the population with Internet access will continue to increase, the issue of valid coverage is an important consideration when conducting a Web survey.
This article reviews Web survey software from three providers. Each piece of software was used to create a simple Web-based survey. In conducting this review, the author imposed several constraints:
- The same short questionnaire was used in all cases.
- The survey was created by the author using each piece of software, and hosted on a site not affiliated to the software provider.
- Each survey was tested to ensure that respondents could complete it online.
- Each software program was used on the same computer (running Windows 98).
The software packages reviewed are: SSI Web (Sawtooth Software), The Survey System (Creative Research Systems), and Survey Pro (Apian).
There are several features that are common to all software packages. First, they all provide a detailed instruction manual that is surprisingly easy to use and follow. Second, they all have very friendly technical support that is available without charge over the phone. In each case, I contacted technical support and asked questions about actually hosting or publishing the survey, and received courteous, helpful advice. Third, all packages are relatively similar from a functionality perspective; they all allow skip patterns, optional response verification to ensure that certain questions are answered or that the response is in an appropriate range, and finally they all provide the ability to show answers to previous questions on the screen (for example, "You mentioned that the last quick-service restaurant you ate at was Burger King; how satisfied were you with their speed of service?")
SSI Web
SSI Web from Sawtooth Software is an excellent Web survey software package. Although expensive compared to the others reviewed - $3,000 for 50 questions, $9,000 for the 500-question version - it is relatively easy to use. A survey is created using either a template or by pointing and clicking and typing in text. It is very easy to change the order of questions or to add graphics.
One of the strengths of this package is the format for laying out and displaying questions; you simply enter your questions and possible responses, and Sawtooth Software handles the formatting and questionnaire layout, while providing the ability to change this formatting if desired.
Of the three packages reviewed, SSI Web also has the strongest survey preview feature, which opens the survey in a separate window and shows how it will actually appear to potential visitors. This is extremely useful to adjust any formatting or page breaks.
 |
Another strength of this package is the online data management module. It allows a researcher to not only monitor responses but also to calculate frequency counts providing a convenient method of monitoring the progress of the survey. Additionally, a researcher can also provide a password to a client, so that they can tabulate frequencies, but not provide the ability to change or adjust password quotas (the read/write capability requires a separate password).
A separate add-on module integrates SSI Web with Sawtooth Software's powerful conjoint packages.
The Survey System
The Survey System is an extremely versatile collection of software packages. With the Basic Edition ($499), users have a complete survey creation and analysis system for questions with 10 or fewer answer choices. Questions are constructed using drop-down menus and help wizards that are common to many Windows-based software packages. The Basic Edition of The Survey System is designed to appeal to all users; those outside of the traditional market research departments can easily create a simple survey, while market research professionals will appreciate the added analytical tools available from an add-on statistical module.
 |
For larger surveys and more sophisticated users, The Professional Edition of The Survey System is reasonably priced at $999. There are several unique features to this package, including the ability to calculate values from other questions, combine Web responses with other information from the same respondent (such as historical data), and the flexibility to establish different quotas for different groups of respondents.
One of the nicest features is the built-in spellchecker. For those of us used to having this feature in Microsoft Word, it is comforting knowing that the chance of a typo is reduced. Also, a range of different questions types can be used, including grid-style tables.
The Survey System has two main strengths. The first is the ability to produce professional, camera-ready reports (complete with tables, charts and text) on a Web page. This feature demonstrates the advantage of Internet-based surveys - the ability to quickly collect the data and quickly produce quality topline reports.
The second strength of this package is the e-mail integration package. Although your ISP may not appreciate (or in some cases may even forbid) the sending of bulk unsolicited e-mail, this feature does provide the ability to manage the e-mail invitation process, including the inclusion of passwords in the link so that respondents do not have to enter them, and the ability to send either reminder messages to non-respondents or thank-you e-mails to those who have. This feature will be very useful to researchers who are using a managed database of people who have agreed to participate in future research projects.
A separate optional add-on module lets users record respondents' answers in their own voices.
Survey Pro
The Survey Pro package from Apian software is a complete data collection package that can be integrated with Apian's other modules. The main survey design package costs $1,195, and the add-on NetCollect module is $595, putting the total price for a single-user license at $1,790.
 |
Since this package is part of a total survey data collection system, it allows the user the flexibility to quickly change from a Web-based survey to a paper survey. However, this functionality makes creating and formatting the Web survey cumbersome. Unlike the other software tools, formatting or editing questions (including changing the order of questions) is not easy.
Alternatively, the strengths of this package are first the ability to quickly integrate the respondent data into a finished report. The report produced is based on a template designed by the user, and can be instantly assembled with topline information once data collection is completed.
Secondly, of all the packages reviewed, Survey Pro is the easiest to host. The process of completing the questionnaire and actually getting it up on your Web site is simple and straightforward.
Publishing the Web survey
Although Survey Pro was the easiest, the process of hosting of publishing the survey was more challenging than I expected. Each piece of software requires the uploading of different script files and will likely require the use of a file transfer protocol (FTP), a method used to upload and download pages from a Web server. I used Ipswitch's WS_FTP Pro (available from www.download.com or www.tucows.com) for this. If you have previously published a Web site using Microsoft FrontPage and are looking for a "Publish Web" button, you will be sadly disappointed.
Part of the problem is that specific requirements among ISPs vary. One provider may require a separate setting accepting perl files, while another requires that all files be uploaded to a common location. This has the potential for the user to be left in the middle, caught between the software maker and the Internet provider. The software company would suggest that you contact your ISP if you have problems, while your ISP would suggest contacting the software manufacturer.
In any case, it is possible to publish your own survey. However, if this seems like too daunting a process, each of these software companies offers a fee-based hosting service, where you design the survey using their software, and then for a fee, they will host it on their own servers - an acceptable solution if you do not have the time or inclination to do it yourself, but still want to conduct Web survey research. Contact each of the providers for more information.
Here to stay
Web surveys are certainly here to stay, and will likely become more and more accepted as a mainstream data collection option. Of the three pieces of software reviewed in this article, two are excellent options for the researcher interested in designing their own Web survey. SSI Web is an easy-to-use but powerful survey design and hosting package that provides all the tools necessary to conduct Web surveys. The Survey System is also a superior product that is easy to use and not only creates Web surveys but also provides the tools for quick and professional analysis and reporting all in one package. I would not hesitate to recommend either package.