Editor’s note: Tim Macer is a U.K.-based independent specialist and adviser in the use of technology for survey research.

As the Internet matures as a means of collecting data, so too does the specialist software for Internet-based survey research. The two Web survey packages reviewed here - Surveywriter and WebSurveyor - are both aimed at professional researchers and provide support for a wide range of Web-based research activities, but they both go about it in radically different ways.

WebSurveyor: a serious research tool with a friendly face

WebSurveyor consists of a locally-installed PC-based survey client that integrates with WebSurveyor’s hosted services on the Internet when it needs to. This means you do not need to be online the whole time you are creating a survey or analyzing the results, and all of the menu items and options work at the speed of a normal desktop application.

A workflow menu of activities is constantly visible to the left of the main work area, which makes it very easy to find your way around the program. A dozen different activities are grouped into three sections representing the three stages of a research project, from writing the questions to fielding the study (in this case called “publishing it,” which is more like switching it on) and analyzing the results, where the program provides some very rudimentary analysis capabilities.

Click on any option on the left and the main window changes to a form where you specify the options for that activity, giving it a Web-like feel, though it is very much a desktop program. A particularly strong feature of the program is the library of pre-defined surveys.

As delivered, it is much more likely to appeal to those working for organizations suddenly handed the responsibility for doing a survey rather than professional researchers. Nonetheless, the library can be developed to add your own particular questions or types of surveys.

There is a parallel feature for defining standard sets of answers, called “response templates,” so you should never have to type “strongly agree” or “18-24” again. This has another benefit, in that it allows you to manage common lists of answers over time. A new brand added to a list of brands will replicate across all questions using that list; it is a feature many survey programs could do with. Templates for the look and feel of the survey work in a similar way, but tellingly, I could not find a feature to switch off the blatant hyperlinked plug for the manufacturer that appeared at the bottom of every page of my survey.

Seamless integration with the Web

There is a useful choice of built-in question types for handling most types of data, including ratings and grids, and you can choose whether you want all questions to be on one scrolling screen (the default) or on separate screens, or any combination, simply by adding so-called page breaks. Page breaks play a crucial role in achieving routing or skip logic, and while there is a lot of flexibility in the kinds of conditions you can set, it was not easy to understand the routing structure once you had created it, which would make writing a complicated screener section difficult and modifying it later positively treacherous.

The program supports multiple languages, though entering them was a seriously tedious cut-and-paste process with no easy import route.

Publishing a survey could not be easier, and here the integration between the desktop tool and the hosted service on the Web works very smoothly. There is an adequate built-in tool for generating personalized e-mail invitations, including some basic authentication methods on query strings or cookies, but no password protection as yet. In addition to WebSurveyor’s hosted service, for which you pay according to several volume-based thresholds, you can pay for an enterprise solution and run the surveys on your own Web server.

The program runs out of steam on the analysis side. The latest version (4.1) adds much needed support for producing summary statistics on numeric and scale questions, and there is the standard gallery of chart views, plus one kind of crosstab. For new users, it is nice that the analysis pops open with a ready-made report showing a chart for every question. But only being able to view each question once, either as a chart or a table, soon becomes a frustration. For serious work, you would need to export the data out to SPSS (which is relatively painless) or something else, which is more hit-and-miss.

Rich in features

Overall, the program is surprisingly rich in features. Its simple, uncluttered interface keeps sneaking in options for the power user on buttons and tabs marked “advanced” and on right-click menus. But these only go so far: advanced capabilities - like introducing randomization at a question level, prioritized selection, and using logic to remove inapplicable answers from questions - are not supported. Neither is quota control, which is surprising. While the documentation is very limited, its manufacturers deserve praise for the informative materials placed on its Web site to encourage good practice in survey design and execution.

Steve Johnson, vice president at Pragmatic Marketing in Scottsdale, Ariz., uses WebSurveyor for an annual market measurement and benchmarking study that the firm uses in its consulting activities with its largely high-tech customer base, as well as to do more general research among its customers. “I am not a researcher, so what I really appreciate is that this handles all of the mechanics, so I can focus on my survey and my answers,” Johnson says. “When I started using it, I was astounded by how simple it was. And over the years the program has gotten more capabilities that make the product increasingly powerful, but without making it any more difficult.”

Johnson contrasted the drawn-out research projects he used to commission 10 years ago for upwards of $50,000 with those he can field for a few hundred dollars using WebSurveyor. “The amazing thing about being Web based is that I can get the answers in three days, and quickly share the experience. It takes about a minute to post the results for others to look at.”

Surveywriter: the can-do program worth getting to know

Surveywriter is delivered entirely as a Web-based service or ASP, so you log on to the company’s Web site and create your surveys there. There are many ASP-based Web interviewing tools now, but Surveywriter is particularly interesting for the balance it strikes between reasonable ease of use and advanced capabilities.

Because it is Web-based, the performance you enjoy or suffer will partly rely on the speed of your Internet connection and the vagaries of bandwidth contention. Surveywriter uses the same server for survey design and live survey hosting, and understandably gives priority to respondents online. At peak times, this could also mean sluggish performance for you when writing your questions. With an open Internet connection required at all times, you won’t be able work on your survey on your laptop on the plane or in the departure lounge (unless you are sitting in a wireless hotspot).

Finding your way around the program may be a little daunting at first. The basic act of creating a new question or editing an existing one relies on moving in and out of a lot of different pop-up windows, some of which have a Microsoft Word-like look and feel. But at each stage, it is easy to preview exactly how each question will look on-screen.

The program considers each question to be on a separate screen, unless you take steps to group questions together into a so-called “merge question.” There are some very sophisticated question types that allow you to do some standard survey activities, such as the one to present awareness or usage questions and then ask follow-up questions only on those mentioned (or not mentioned). These go by the curious name of “piping” and “regressive piping” in the tool, which can be forgiven as they make a tedious programming task into a very simple one.

In all, there is little that has been overlooked in supporting advanced survey design. There is built-in support for quota checking and for screening and terminating interviews part way through. Standard message screens handle each of these eventualities, which you can customize on a survey or on an entire account basis. And since you can set up multiple accounts at no extra charge, and link the accounts together, you could use this to create different strategies for different client groups. Templating also works at both a survey or an account level, and there are seemingly endless capabilities in presentation options.

Maximum security

The suite offers a range of security features to keep bounty-hunting respondents at bay. This includes excellent, integrated capabilities for generating personalized e-mail invitations containing click-through links or password information. If you do not wish to upload your sample to the ASP’s server, and prefer to do a mail-merge directly from your own database, the program will helpfully generate unique passcodes or PINs on demand, which you can download. Use them with your own mail-merge or database and you will still have security and traceability in your research.

Translating surveys into multiple languages is also well supported, with the ability to spin out all the English text into an Excel spreadsheet for the translator to work on offline and simply to re-import all the translated text.

For really advanced users, there is even the facility to write your own Java applets between questions, to interrogate or update other databases in real time or perform other non-supported activities.

The program works on the Internet principal that it is free to use, until you need to do something useful, which in this case means to collect responses. Here, you pay only for completes. This fee structure also gives you access to some basic online reporting, but for an extra fee you can spot-purchase access to a more advanced analytical module on a survey-by-survey basis. Yet, for such a vast and comprehensive system, its documentation does not appear to do it justice and also lacks good examples. A better manual would certainly result in fewer support calls to the supplier.

David Taber runs his own research business, Taber Analytics, in San Francisco. He was initially a little unnerved by the prospect of placing precious and often confidential data in the hands of a total stranger, yet he reports no problems to date. “It is very reliable and robust,” he says. “I have just received 1,200 completes on one study and I don’t think I had a single e-mail saying the survey did not work. I’d normally expect about 1 percent to report some problems.”

Taber had worked his way through several similar packages, each time running up against limitations, before finding Surveywriter a year ago. “I tried a project with it and it worked pretty well. I spoke to their technical guy, asked some difficult questions and he seemed willing to get his hands dirty. It has been a great choice for me. It provides a good level of power, given how much it is costing you.”