Data Use: High-tech surveys have arrived

Abstract

This article reviews three software packages for developing and delivering questionnaires: Sensus Media, Decisive Survey and Survey Said. Some of the features highlighted include integrating multimedia components, incorporating earlier responses into later questions, handling various calculations with the data, and delivering questionnaires through various methods (Web, network, e-mail, etc.).

Editor’s note: Steven Struhl is vice president, senior methodologist, and Chris Kuever is vice president, director of data processing, at Total Research, Chicago.

The fine art of writing interviews is getting more complicated. As in many other areas of life, we have computers to thank for this. It seems that interviewing, any minute now, will step onto the World Wide Web in a big way. (Have we heard this enough times before? This time, though, it seems true.) Other new software will allow your PC to do interviews with all sorts of striking multimedia abilities. High-tech surveys have arrived.

For those of you who just mastered a CATI system, or are still struggling with one, or want to stay with paper-and-pencil interviews, these new technologies may not sound like the greatest news of the decade. Fortunately, PC software has become more intelligent, not just more complicated. Using new software can be quite a different experience from what you might have expected just a few years ago. Now you sometimes can start using something both new and technological without much of a learning curve (i.e., a period in which approximately nothing works and you live in non-comprehension and terror).

You may find the best of new software, even if it requires you to pick up some new concepts, is surprisingly intuitive. ("Intuitive," another term with special computer connotations, means - more or less - that the software does not behave in remarkably unexpected, bizarre, or repellent ways too often.) In some cases, you can nearly get new software to work right out of the shrink-wrap. Given the recent history of high-tech, that’s amazing.

We will discuss three software packages in this review. Because PC applications, and the Internet in particular, are burgeoning, you can find many, many other applications that try to do the same things. It is more than possible that we have missed your favorite application, or some promising new contender. These three applications really work, though, and have proven themselves under fire with extensive in-the-field use. Below, we have a brief paragraph on each, and then we will get on to the main reviews.

Sensus Multimedia by Sawtooth Technologies is (as the name suggests) the multimedia PC-based package. For those of you not versed in the new terminology, multimedia programs (if they are really multimedia) can use sounds, images, and even little animations or movies in addition to standard text. Sometimes, you will see a program called "multimedia," even if it can do no more than play a simple Windows ".wav" file (like the one that says, "Welcome!" every single time you start America Online). Fortunately, Sensus Multimedia does much more than that - as we will see below.

Decisive Survey by Decisive Technology allows you to do interviewing over the Internet and by e-mail. The program handles both creating the surveys and gathering the data.

Survey Said by Marketing Masters does interviewing over the Internet, like Decisive Survey. Also, like Sensus, it allows you to do various types of PC-based interviewing.

Sensus Multimedia

Sensus Multimedia (version 2.0) shows how detailed knowledge of interviewing and research practices can help make a better interviewing package. Sensus comes from Sawtooth Technologies, a firm with long experience in market research (some of you may be familiar with their Ci3 CATI interviewing package). Version 2 packs nearly all the question types and question sequencing options that you are likely to need. It lets you lay out an attractive PC-screen-oriented questionnaire, using multiple fonts, pictures, sounds, and even animation or movies. You can run the resulting questionnaire on a PC, or send it out as a disk-by-mail survey.

Many useful features

Perhaps the quickest way to give you an idea of Sensus’ capabilities is to show a checklist (see below), with the features new to Version 2.0 highlighted in italics, and with a special symbol.

Some terms in the checklist may require explanation, especially among the newer features. Those of you who already know all the latest terminology can safely skip to the next section.

Under question types, we have several terms that may require explanation:

"Analog rating scales" means a sliding scale or slider, like the one in Figure 1
A "roster" is a page in which a respondent evaluates a series of items on the same rating scales, but is not a "grid" (discussed right after this). Rather, items rated and the scales on which they are rated both are rotated. The program displays each combination of items and rating scales, and then goes on to the next. These go by one at a time on the computer screen, and so could take a while for a long list of items rated on many scales.
A "grid" is more familiar, looking like a table in a word processing program, with the items evaluated as the row headings and the columns as the items or scales on which these are rated. The example above shows how a grid might work.

Under response options, "radio button" means a small circle that changes from hollow (white) to black when clicked. It differs from a (plain, unmodified) "button," which is a small object (round or square) that you push on the screen. These terms are, by the way, standard Windows terminology - so if you did not know them before, you have just learned something crucial to your ongoing existence.

The various file formats are used for sounds (WAV files), still pictures (BMP, WMF, WPG, DIB, JIF, JPG, PCX, TGA, and TIF), or pictures that move (AVI, MPG and MOV). If nothing else, these show not only that Sensus is versatile in handling file format, but also that Windows has not brought with it any standardization of how data gets stored. Interestingly enough, Sensus will not play the animated GIF files that are becoming so popular (and annoying) on Web sites these days. We are anticipating ourselves here; a discussion of the Web is yet to come.

Some excellent features

Among Sensus’ many fine features, some of the best are found in its abilities to incorporate earlier responses into later questions. It even can do calculations that follow through to later answers. For instance, suppose you fill out a survey on tractors, and say that you have two Fords, three John Deeres, and four International Harvesters. In a later question, the package could then ask you "Thinking about your nine tractors . . ." It could then go on and ask you questions about your two Ford tractors, your three John Deeres, and four International Harvesters. In short, it remembers what you said, it can add, and can come up with the appropriate phrase at the appropriate time. It can even multiply answers, or use them in calculations with parenthetical logic - and show the results to the amazed respondent. This is pretty smart for a computer.

Working with Sensus

You almost certainly will satisfy nearly all your survey-related needs with the basic Sensus Multimedia package. Sensus also can include its own programming language module, Sensus ADE (at an extra cost), and this allows you to extend the capabilities of the package even further. Sensus ADE looks somewhat similar to Visual Basic. So if you know something about Visual Basic and you like programming, you should find this familiar and perhaps even congenial to use. (We will be discussing the basic multimedia program here, not the ADE language.)

Sensus appears as several programs, each of which handles one aspect of its operation. These parts are called:

Figure 1 shows the Sensus multimedia "Create" program, which is where you put together the survey, and where most users likely will spend most of their time. As the figure suggests, quite a bit can happen in this part of the program.

This screen shows you some of the many types of questions and "objects" you can build into a Sensus interview. The area with the gray background corresponds to what the respondent would see. To the left is an actual movie, in the AVI format. With typical video efficiency, it uses about 796,000 bytes for a whole two seconds of playing time. (This use of storage space is something to think about if you use Sensus for disk-by-mail interviews, since these interviews must go on a single floppy disk. Images or sounds that would require more than one floppy disk therefore, cannot go on disk-by-mail surveys.)

On the right side of the gray area, you see three types of questions. The topmost is a nice analog "slider," which records responses as numbers based on the position of the small sliding bar. You would need either to add some special Sensus text, or to import a picture from another program with a numeric scale on it to get numbers next to this control. Sensus can resize and move pictures from other programs, but it cannot create or edit them. The image needs to look right before it gets to the program.

The next control below is a scrollable numeric, or "spinner," which goes from a specified minimum to a specified maximum value. The respondent uses the up and down arrows to change the answers with this control, or can type in a number.

To the right you will see a "property" window. The properties here refer to this control (the spinner), which you can tell (more or less) by its appearance - a small box outlines the control itself, which we selected on the screen with a mouse click. You will see in the "property window" that you can set many characteristics of this control, including its font, color, and the minimum and maximum values that it will accept.

To get this "property" window to appear, you can call it using the "Window" menu near the top of the screen or the F4 key. This differs from many Windows 95 programs, in which you can get the "properties" of an object on the screen by clicking on it with the right mouse button. This does not happen in Sensus yet, perhaps because the program also runs under Windows 3.1.

The lowest control in the gray area (which corresponds to what the respondent sees) is called a "masked numeric field." This means that it will not accept any answer above a certain value. This one will take any number with no more than three digits. We added appropriate instructions for those who want to enter values of 1,000 or more. Incidentally, another nice touch is that you can set the masked field to accept only a certain numeric format, like dollars, or percentages.

To the right of the gray area, and below the property window, is another small window that shows you where you are in the questionnaire. This allows you to navigate from one page to the next easily.

At the bottom of the diagram, you see two menu bars, one labeled "edit" and the other (with a title much harder to read in the figure) labeled "objects." Only one of these bars can be active at a time, which explains why the objects menu is less visible - at the time of the screen capture, it was sitting there inactive, or inert. The edit bar is for text only. So you either can edit the text on the screen, or the other "objects," such as the controls, at any one time.

Sensus has other fine features that do not appear on the screen. For instance, it lets you check the questionnaire logic and skip patterns starting at any point in the questionnaire. You can go directly to the section you need to test, and run everything from that point forward.

Some problems: a few thorns on the rose

Sensus 2.0 does some marvelous things. Still, a few things about it just did not work quite right. Nothing led to irremediable problems, but Sensus caused a few irritable moments. Let’s start with the more vexatious of these.

We cannot say that installation went without a hitch. We tried to install the program on the E:\ drive of one of our PCs, which did not work well. The installation ran, and the program appeared, but it kept looking for files it needed on the C:\ drive. A call to technical support revealed that Sensus in fact will only run properly if installed on drives designated as either C:\ or D:\. The program documentation did not mention this.

Sensus comes on a set of six floppy disks, so reinstalling was not entirely painless. Limiting installation to only the first two drive letters is one restriction that the makers of Sensus need to eliminate.

Also problematic to us was that critical elements in the questionnaire disappeared using a standard monitor resolution and set-up. In particular, using the "large fonts" option (which was preset in the Windows display panel) made two key questionnaire controls disappear. The items that vanished were the "Next" button and the "Back" button that appear in the questionnaire, and without these you cannot go from one page of the questionnaire to the next.

Eventually, Sawtooth found a way for us to get these buttons back on the screen, but it was not readily apparent. Before we found out about this we fell back on changing the Windows screen display set-up to use "small fonts" instead. As we will describe, this could make you very irritated.

There was no warning that parts of the questionnaire could vanish if your PC uses "large fonts." Sawtooth provides a nice, thorough-looking description of screen resolutions in their manual, but somehow this piece of information did not make it there. (We would strongly suggest that Sawtooth add some warning about this problem. Even a small sheet of paper slipped into the front of the software manual would help.)

Anyhow, at first, it just seemed that the program would not work. No control buttons on the screen would mean that respondents never could get past the introductory page. This could lead to some highly undesirable results if you try to use the program for disk-by-mail surveys. Almost certainly, most people who had a problem like this would, at best, throw the survey away. You also might hear some language over the phone from them that you genuinely would not appreciate.

Unfortunately, without Sawtooth’s secret work-around, dealing with this problem was a nuisance. You will need to restart Windows to change from "large fonts" to "small fonts." (You need to do this even if you have that nice Quickres software gadget that Microsoft gives away with its PowerToy software bundle. Quickres lets you change the resolution of your screen display without restarting Windows, and you can get it from Microsoft’s Web site -- and yes, it and the entire PowerToy bundle are absolutely free.)

One of us (Struhl) really dislikes small fonts. They make Windows menus and much of the other text on the screen much harder to read. So switching to them from the big fonts has a high nuisance quotient.

Unfortunately, Sawtooth’s work-around also requires a lot of work. You need to find the object (like the "next" button) listed in the "properties" window, highlight its name, and press the "Control" and "Home" keys together. The control jumps to a place in the corner where you can see it, but probably not where you want it. You will need to repeat this and drag the control where you want it on each questionnaire page.

It would all be much easier if the program worked with large fonts in the first place. This one aspect of Sensus seems more than ready for an upgrade.

Aside from these two problems - which need fixes - the program had a few other areas in which performance could use a slight boost. Oddly enough, Sensus seemed to behave less flexibly in its handling of text (i.e., questions that appear on a page) than in its handling of the various controls and objects that you can add. (Sensus actually has two kinds of text, but the kind that is easy to move doesn’t have flexibility in fonts and colors, and vice versa.) The text that you can format best finally will go where you want it, with a little work. You cannot push and pull these nicer-looking text "blocks" using the mouse, as you can with nearly any drawing or desktop publishing program. You need to set the size and position of this text with a dialog box that you call up from the edit menu. Figure 2 shows the dialog box that you need to use, to set where this text appears.

Anyhow, having two different types of text, and making the nicer one (that you will want to use) require this dialog box, makes formatting a little cumbersome. Unfortunately, the nicer text also tends to "bump" the other objects (controls and buttons) on the page when it is moved or changed, so lining up things on the page required a fair amount of work.

Overall, this part of the program seems to need a little fixing. We would like to see Sensus make text into one type of object, and make that work like other objects, so you can format, maneuver, and resize everything on screen, using a mouse.

We have one final caution about sending out disks to field services. It you have multimedia objects, expect the interview to run across several floppies. Even without small movies included in the interview, your survey likely will take up several diskettes. The program will "install" these, along with a Sensus "field" module to run the survey on another PC. If you include a batch of pictures, a movie or two, and some narration, you will wish that PC makers had long ago standardized on the nice 100 MB Zip drive, or the nearly as nice 120 MB super-floppy. You may end up swapping in quite a few disks for the survey with each holding a mere 1.44 MB.

Sensus overall

Sensus Multimedia costs about $2,000, and is well worth the investment if you need strong multimedia interviewing capabilities. It is powerful and full-featured, and does many remarkable things. The makers of Sensus obviously have applied long experience in market research to the design of the program and its features, and it shows. Grumbles about the program reside in two principal areas: problems with the video display that required resetting Windows’ basic display properties, and some awkwardness in moving the question text in the display. We hope that Sensus will work on these areas -- and allow you to install this product on a drive with a letter higher than "D." If Sensus can resolve these problems, it will really deserve that title the programmers love the most: a "killer app." Even with a few things left to resolve, Sensus is the best PC-based interviewing package we have seen yet.

Internet and World Wide Web: a brief primer

Before we start with our Internet interviewing packages, we’d like to give you a brief review of the key terms. If you are one of those lucky people who (think they) know everything there is to know about the Web and the Net, you can skip right to the next section. For the rest of us, let’s start with the basics.

The Internet is a huge data network that spans most of the globe, linked by an enormous assortment of high-capacity data cables, over thousands of routing locations. Data that travels over the Internet get broken into small pieces, or packets, which get routed to their destination and then reassembled. They share their data pipeline with untold other packets, all going to their various destinations. This makes the Internet highly efficient, because it keeps the pipeline as full as possible at all times. (The routers are computers that figure out where the packets should be sent to keep things flowing.) This is very different from the transmission methods used by your friendly telephone company to connect your phone calls, where you get an entire line dedicated to your conversation, included the pauses and blank spaces.

This is all very impressive and wonderful technology, even if it does seem to get a lot of use for junk e-mail, going to the wrong places, aimless browsing, and downloading dirty pictures. (Not by any of our readers, of course.)

The World Wide Web is simply the component of the Internet with a graphical interface. For those of you who are not technically inclined, that means it is the part that looks a lot like America Online. Also, you will need a Web site of your own (or will have to borrow a friend’s) to run either Survey Said or Decisive. Responses must come back to a specific location.

The Internet truly is growing explosively. For instance, a company that you most likely never heard of is just now finishing up a data "backbone" across the U.S. that will have roughly 20 times the bandwidth (or capacity) of AT&T, MCI, Sprint and WorldCom combined. (By the way, the company is named Qwest. Didn’t we tell you that you never heard of it?) No doubt all this capacity will soon find many new uses carrying data, much of it on the World Wide Web.

As another "for instance," data transmission speeds available to everybody are about to increase dramatically. Microsoft, Intel and the various Baby Bells (assuming more than one of them remains after they finish merging) have already cooked up a new and promising transmission technology called ADSL. This will work on current equipment, including copper phone wires (they say), and run anywhere from 20 to several hundred times as fast as your modem does now. Of course, the Baby Bells are the same crowd who took a promising technology called ISDN and made it so expensive and horrendously complicated that they managed to scare away nearly all potential buyers. Still, Microsoft and Intel, which at least know how to make products that people will buy, may have enough to say about ADSL that it will work reasonably well. If not, there’s another competing technology right behind it.

The conclusion then, is that the Internet is here to stay, it is getting more important all the time, and is likely to get much more important in the future. The two products that we will discuss next will let you hop on board, creating surveys that can go out on the World Wide Web in style.

Survey Said

Survey Said is a highly versatile program, as its opening display screen suggests. You can create PC-based surveys and disk by mail surveys, as you can with Sensus. You also can make scannable paper forms, and develop surveys to send over a network. Finally, you can go on the Internet using HTML formatting or Java "applets."

HTML, for those of you not filled to bursting with the latest Web terminology, stands for hyper-text markup language. This is the format in which most Web pages are created, and it is both versatile and powerful. HTML allows you to create so-called links (which whisk you to other pages when you click on them - or at least get you to them after the inevitable lengthy pause). HTML also allows you to embed pictures, sounds, and movies right into a page you create for the Web. (We’ve slipped this into this section just to get all of you who skipped the Internet primer earlier).

Anyhow, Java applets are one of the newest things on the Web. Java is a new programming language that is supposed to work on every type of computer imaginable (not just machines running Windows). It is very compact, so when you call up a page, it can transmit Java "applets" along with all the text, pictures and sounds. As their name implies, applets are very small programs with specialized functions (and not small apples, as their name also could imply). In theory, these little programs can make the Web page behave as if it had some intelligence. At least they would if they always worked, but since they are so new, they seem often not to do so. To get results, you usually need to have the latest in Web browsers, and maybe the right PC, and maybe the right phase of the moon or astrological sign. Unfortunately, Sun Microsystems (and not Microsoft) is the driving force behind Java, and so it may never really catch on everywhere. (Microsoft, by the way, already has developed specialized versions of Java that run well only with Windows - so there goes the promise of a programming language that works universally.)

Fortunately, as we mentioned earlier, the HTML capabilities of the program give it all the power you need to get good-looking surveys on the Web. In fact, with HTML you can make your Web surveys into multimedia extravaganzas, almost as nice as those you can make with Sensus.

Interestingly enough, Survey Said does not say much about these capabilities, in either its manual or (stranger still) in its advertising and promotions. In fact, Survey Said can do more in the area of multimedia than its manufacturer (Marketing Masters) tells you. Admittedly, you need a package that can edit HTML Web pages (like Microsoft FrontPage or Adobe PageMill) to insert all the wondrous multimedia stuff. Still, neither of those packages is expensive, and the surveys you create will work no matter what you add later. The manufacturer of Survey Said needs to blow its own horn about this a little more (and this is the first time we can recall ever saying this about any software manufacturer). It also would help to have a section of the manual devoted to ways in which you can add multimedia.

Survey Said allows you to create and modify survey content in a program it calls the "Survey Librarian." Once you call up the survey, you get an editing window, which provides you with all the question alternatives available.

Figure 3 shows the library window with a survey selected and with the "modify" option already selected. The list in reverse type shows you the choices of question types. You have a wide variety, including rankings, ratings, scroll bars, selecting any of several answers, and point allocation.

Sensus has a few more question types, but you should find Survey Said adequate for most applications. About the only complaint here is that the formats are somewhat inflexible. For instance, rating choices must start with "1" as the lowest number (rather than "0") as some of us favor. You will note that in the illustration (Figure 3), we needed to use a question format that put the letters "A" to "K" next to the choices.

We need to mention two points here. The survey form that you see on your screen looks like the version that will run on a PC or via disk-by-mail. This does not look exactly like the version that would appear on the Web. In the Web version, everything appears as a long page in HTML format, and the choices become either "buttons" (radio style or plain) or pull-down menus.

The letters "A" through "K" that you see in the editing screen here will not show up on the Web survey. In the PC version though, you cannot get rid of these letters or change them to anything else. Using these letters can help with data entry (you can punch these in quickly), but it’s a shame they can’t be varied. We have some long-standing evidence that using the letters "A" through "F" to label questions can introduce bias in responses.

Also, this makes it problematic to run surveys on both the Web and on PCs and have them remain truly compatible. If the surveys do not look the same in both versions, we can’t promise that responses from both will be strictly comparable. In any event, a little more flexibility in labeling the various buttons, and in setting up scales, would only be welcome.

Still, you will find that the program offers you many options in its preset forms. If you can live with the program’s small inconsistencies, you should be able to find something that gets close to what you need in any particular question. (For instance, in the question type shown in Figure 3, you can have anywhere from one to 24 answers - although it would seem you need not have the option of selecting one choice if only one is possible.)

Survey Said cannot do calculations and feed sums from your earlier answers back to you. You can use detailed skip patterns to guide respondents to different questions depending on their answers, though. Figure 4 shows you the window in which you can edit skip patterns. Note that you can select a different question as the point for skipping for each possible answer.

In one moderately confusing aspect of the program, you only can get to this window if, under the main "File" menu, you select "Open survey" rather than "Open library." It would be much simpler if the menu said something like "Edit skip patterns" to direct you toward the window where you can do this. As shown in Figure 4, you can see and edit all the skips in one place, making it simple to check the logic of the survey.

More about HTML

Here we need still more explanation of HTML. Once the survey gets created, you will need to translate it into HTML format to run it as a well-presented document that behaves itself on the Web. As things now stand, you get precisely one version of the HTML-format survey, and all the questions must appear on it. Skip patterns in the computer-based survey become instructions in the HTML form. At the risk of running far too long an example, we have included below a few sample questions that worked nicely with skip patterns on the PC transformed into HTML format.

By the way, to see the survey in this form, you can just call it up from your hard disk in your Web browser, such as MS Internet Explorer or Netscape Navigator. The excerpt below looks much like a part of the Web survey. We pasted it with no trouble into this document, in the word processing program, going directly from the Web browser window.

Survey Said added all the skipping instructions you see, but you will notice that this just is not the same as having the question disappear when the respondent does not need it.

As you can see, certain types of skipping and branching that we more or less take for granted with other survey forms are much more of a problem here. Putting in some forms of question control that are easy to administer elsewhere - such as rotating questions, or having some respondents skip certain blocks of questions - seem really difficult on the Web. With Survey Said, you would need to create several versions of the survey, by hand, and then reassemble the answers from them later. This seems like the type of work the software could do, but this is not the case yet. And there seems to be nothing you can do to whisk respondents by questions they do not need to see. In some ways, in spite of the slick graphics you find on the Web, we are still roughing it there, after all.

Perhaps at some time in the not-too-distant future, Java applets will solve our problems here. Survey Said can generate a page using the latest Java tools, which means that you can have the type of control you want. Unfortunately, as we mentioned earlier, Java applets are so new that today’s Web browsers may not run them properly. We will need to wait, to see if Java becomes the truly standard language that it needs to become, if we are to use it with Web interviewing.

Survey Said today: the program runs everything from one file

Survey Said has a special method for creating a ".cgi script" that will work with all its different formats. Again, the versions may not match 100 percent, but at least, once you get the survey working, it will work on the Internet, on a PC and with disk-by-mail. So you do not need to rewrite the survey several times to get it to run in several places.

The program uses a nice system on your Web site, with which it routes the answers to the correct survey. You can have as many surveys running as you want, and the program will simply pull out the answers from each and send them to the correct data file. It has several built-in security features as well. These prevent amateur hackers from making system accept anything but approved answers to the survey, and prevent any damage to the main survey-collection file.

You can customize the messages that Survey Said will send back to respondents when they complete surveys, as well. If you have several surveys going at once, you can have a message of receipt or a "thank-you" specific to each.

If we have not said so, the PC version is a full-scaled program in its own right, not a simple add-on. It has many fine features of its own. Among them, it stores the survey compactly so that you can have up to 300 questions on a single floppy disk -- far more than even the most tolerant respondent will suffer. You also can set the program to accept one respondent or multiple respondents (up to 200, depending on the survey’s length) on a single diskette. This could simplify sending the diskettes out to a field service. Finally, the program does not need to install anything on the PC to run the survey. Everything you need comes right on the survey diskette - and stays there.

Survey Said comes in three versions with different limits: the two lighter versions cost a few hundred dollars less and are limited to 400 and 700 responses each. It is really worth spending the extra money to get the full version, though, as this will allow you to collect as much data as you want for not too much more expense (it costs $1,000).

The program should run with nearly any type of Web-server (that is, the computer that runs the Web site). If you are running the site from Windows, you get the needed executable files with the program. If you are running the site from UNIX (as many do), Marketing Masters provides the source code so that you can compile it to run under whatever flavor of UNIX you have. (UNIX is somewhat picky about how a program’s code gets compiled, each version wanting is own compilation.) If you have a Web master, setting up SurveySaid on your site should be a simple task for her or him. If you are new to the Web, and are doing it yourself, just take a few deep breaths, and follow the instructions. You will find them clear and thorough.

Some small rough edges

Survey Said looks a little less slick than either Sensus or Decisive (coming up in the next section), but don’t let this appearance fool you. Survey Said is remarkably versatile and does its job. We have some complaints about it, but only a few. For instance, installation leaves a little to be desired. The program automatically creates its own Windows "group." (In Windows 95, this appears as another entry in the main "Programs" menu - the one that you get by clicking the "Start" button). Most programs now have slightly better manners, and ask you if you want to create a new group, or install the program in one you already have.

Also, we found a slight "hole" in Survey Said’s PC-based questionnaire versions. If you allow respondents the option of reviewing their answers (which you can deny them by simply changing one check-box in the survey set-up), then they can back up and answer questions that they were supposed to skip also. We’re not too sure what this does to the data file, but clearly if a question is supposed to be off limits, it should be off limits.

Since the Web version relies on one long HTML page, all the questions sit out there in view, whether respondents need to answer them, or not. Maybe once the Web gets faster, answering a question in a certain way could send you, via a special link, to an entirely different page with the appropriate question. But for now, one big page seems like the only way we can go.

Otherwise, we have a "wish list" for, rather than gripes about, Survey Said. One salient omission is its lack of a spelling checker. You can export the questions to another file and read them into MS Word or some other application, but this takes more time and effort than it should. It seems to us that any package that generates self-administered questionnaires (that your critical public will view) should have a spell checker as a standard feature. Anyhow, if a program that you can find for about $39, like Micrografx Windows Draw, can have a spell checker, it can’t be that tough to add one. We hope the point is taken.

We would like to see the program become more flexible in the ways it displays choices on the PC screens, and we would like it to become more sophisticated in its use of branching and skip patterns there. Even if this option must remain limited on the Web, and rotation means building several alternative versions, we would like to see these capabilities taken further.

For the PC version, it would be helpful if it could pick up earlier responses, and incorporate them into later questions, as Sensus can do. Also, when you run Web surveys and PC surveys, they should look as similar as possible. At the very least, the program could do without the extraneous letters that appear with choices on the PC version. Finally, we also would like to see a section of the manual devoted to tips on making HTML surveys into "multimedia" documents.

Survey Said overall

If you have straightforward surveys to do on the Web, Survey Said should provide you with all the horsepower you ever will need. It is a versatile program, and even allows you to send out the same survey in several forms - on the Internet, using disk-by-mail, and even on paper. You have to accept some small differences in format from the PC to the Web, but versions at least will look good. If its capabilities meet your needs, you should find a lot in this program to please you. We found it easy to use, and fast in creating surveys that will work on the Web. Its data collection facilities allow you to get several (or many) surveys up and running on one Web site. At its price of about $1,000, the "full" version (which handles as many responses as you want) is a great bargain for real "leading edge" technology.

Decisive Survey

Decisive Survey offers truly industrial-strength software packages. Their larger version, Decision Source (which we will not discuss here), can process unlimited numbers of surveys, requires a dedicated Web server, and commands a high price. This is software used by enormous organizations, like America Online, that sometimes need to gather millions of responses from their customers. (Although, come to think of it, neither of us has ever been among the millions who must have gotten an America Online survey, either. Just a random observation, here.)

We will discuss Decisive’s "smaller" size, the 10,000 respondent edition, here.

Decisive has some ambitious goals for its product, seeing as part of a "real-time information" system, which can provide ongoing "feedback" from customers to manufacturers or service providers, or from employees of large organizations to their managers.

Decisive Survey seems well suited to these types of tasks. However, it does not give you the variety of question types that you will find in either Sensus or Survey Said. As Figure 5 shows, you can choose among four basic styles of questions:

Choose one
Choose all that apply
Rating
Enter text

Decisive Survey does not allow for scaled responses, except as simple text. It does not do rankings, point allocation, or sliding-scale questions. It allows you to skip to other questions, but does not provide any more advanced forms of branching in the questionnaire. Again, recall that the Web (for the moment) does not fully support many fancier question formats, which will need to wait for widespread adoptions of Java applets to run intelligently.

If you find its range of capabilities meet your needs, you should like Decisive Survey. The program is polished and well thought-out, with a nicely integrated "interface" or appearance. Figure 5 shows you that the layout is compact and intelligent, with everything readily accessible from well-organized menus.

Decisive Survey, unlike Survey Said, gets its responses back in the form of e-mail. Each survey comes into a designated e-mail address on your Web site, rather than into a dedicated file. Because it uses this transmission method, you can easily put Decisive onto an employee e-mail system. Decisive may well be your product of choice if you intend mainly to send out surveys via e-mail.

Like the other products, Decisive can gather the responses for you and do some basic data processing. It knows enough to ignore answers given to Web survey questions that respondents should skip (Survey Said is still working on getting this feature into their package). If you have a Web master, you should find it simple to get Decisive up and running. Otherwise, the manual is clear, well-laid out, and helpful. Decisive, like the others, does some summary statistics, but not nearly as much as a dedicated data analysis package, such as SPSS or SAS. Also like the others, it will export data to these packages, but only as a plain text (or ASCII) file. Unfortunately, also like Survey Said, Decisive lacks a spelling checker.

Decisive in brief

Decisive obviously has the capacity to handle as many surveys as you can throw in its direction. It has a very impressive client list, having met the survey needs of organizations with millions of customers, such as America Online. The program is well-organized and has proven under fire that it will do what it sets out to do. You will need to restrict yourself to four basic types of questions with Decisive Survey. Also, it will not catch your spelling errors automatically. If you do not mind these limitations, you should find that it performs well. Since it relies on e-mail as its basic data collection method, you may find it the best choice if you do mostly e-mail-based surveys. The 10,000 respondent version costs $2,400.

A final comment: talking to and sharing with other programs

Part of the great promise Windows has always had (or is that a promise it has always made?) is that Windows programs would share data with each other. An ideal Windows (software) inhabitant should be able to give over whatever it is working on to another program, whenever needed. Unfortunately, programs that have anything to do with writing surveys typically have stayed well behind the curve in this area. Even very expensive packages, like CfMC for tabulations, only recently have managed to produce files in anything but their proprietary file formats or plain text (ASCII) files.

It is a real waste when programs which allow you to create questionnaires - and so have questions with numbers and labels already on them - but can export questionnaire data files only as plain, unlabelled text. Yet you and I know - and we would hope the makers of these programs would, too - that somebody, somewhere might want to analyze the data more completely than their programs allow. Then perhaps, the manufacturers could make the leap, and allow you to export data in a form where it has labels. You might even wish that they would learn how to export files to SPSS and SAS, the two big noises in statistical software. (If the program could export the labels, then you would not have to put them in all over again to do any other analyses.)

But usually this seems like wishing for too much. Information that you painstakingly enter for most questionnaire-related programs is lost for all others. Perhaps this is part of the hubris that seems to crop up in the software business - where some manufacturers start believing that their products ought to serve as the one-stop solution for all needs. Maybe the makers get exhausted just cooking up their own list of special features. Whatever the case, we earnestly wish that the manufacturers would see the error of their ways, and start to make the software do the real work in exporting files, rather than leaving this to us poor users.

Apropos to this, the head of a large software development team once remarked to one of us (Struhl): "What? You actually expect it to do what you want? Don’t you know it’s only software?" End of complaint, and we hope that the point is taken - at least by someone, somewhere.

Summary

These programs may, indeed, be only software, but they show a great deal of intelligence and many good features. Sensus Multimedia will give you an outstanding solution if you need to do advanced PC-based interviewing. It is flexible, can do complicated skip patterns and logic, and remembers what respondents said for use in later questions. It can sum responses, multiply them, and even exponentiate, or do math involving parenthetical statements - which is more than most respondents can do. You may have to do a little fiddling to get things just right, and you will want to watch for some rough spots in the installation and in how it uses the Windows screen display. Overall, Sensus Multimedia is the best PC-based interviewing package we have seen.

Survey Said is the leader in versatility, and the most reasonably-priced of the programs. You can use it to interview on the Web, on PCs, using disk by mail, and even to create scannable forms. Although it shows a little inflexibility in question formats, it provides a wide range of question types. It does not have all the sophisticated features you will find in Sensus, but then some of this seems inherent in the Web itself. And Survey Said lets you put your survey up on the Internet, which Sensus cannot.

Decisive Survey is probably the program to choose if you will be collecting millions of responses - but in that case you will want Decision Source, the larger, more expensive version. This program only offers four basic types of questions, but it is nicely made, and intelligently designed. It uses e-mail as its basic method for returning surveys, and it seems particularly well suited for surveys sent out in this form. If you primarily do surveys by e-mail, and need to go out on to the Web at other times, this may be the software package of choice for you.