Editor’s note: Scott Young is vice president of Perception Research Services, Fort Lee, N.J.

To develop effective Web sites and measure their impact, marketers need to regularly gather the input and feedback of site visitors. Web site-based surveys, in which people are "intercepted" and questioned as they visit a site, are a fast, affordable way of carrying on a dialogue with site visitors. The following case histories illustrate how surveys are being used to address a broad range of Web marketing issues.

  • A media company wanted to gather information which would assist in the sale of advertising space on its Web site. Over the course of one week, every 50th visitor to the Web site was intercepted and asked to participate in an on-line survey. Nearly two-thirds of those asked chose to respond, resulting in over 2,300 responses. The survey uncovered that a majority of the site’s visitors had made purchases over the Internet - a fact which was used in promoting the site to advertisers. In addition, it found that the Web site was extending the "brand" to a new audience not familiar with the television channel.
  • A publisher wanted to document the value of its Web site in driving off-line book purchases. For two weeks, a screen icon was placed at different sections of the Web site, inviting people to participate in an on-line survey and offering a free book as an incentive. Over 2,900 people chose to take the survey, which confirmed that the site was influencing several million dollars of sales taking place at traditional "brick & mortar" book stores and/or other Web sites. It also revealed that most site visitors were extremely heavy book purchasers (over 20 books/year) - and that many had become aware of the site through off-line advertising. These insights confirmed the value of the Web site in reaching and influencing book buyers - and guided decisions regarding content, design and site promotion.
  • A travel-oriented Web site wanted customer feedback to guide the development of new site features. Over one month, 650 people completed an on-line survey, which asked them to discuss their reasons for visiting the site, rate the site on different dimensions (i.e., content, navigation, etc.) and react to several new content ideas. The survey confirmed interest in a customized travel planning service and provided guidance for its development. It also uncovered a demand for improved maps and revealed that changes in design were needed to facilitate the printing of different pages/screens.

While site-based surveys can play an important role in Web site design and assessment, their value ultimately depends upon how well they are implemented: Poor sampling or survey design can result in misleading findings. With this in mind, here are several guidelines to consider in developing and using site-based surveys.

  • Actively intercept site visitors. To gather the most representative sample of responses, it’s best to actively intercept site visitors (i.e., every 20th or 50th visitor) rather than simply posting an icon/link. If a random-intercept is not possible, consider offering an incentive for participation, yet keep the potential cost of fulfillment in mind - and beware that this may draw first-time visitors in search of giveaways. A link without an incentive is essentially the same as the "leave us your comments" icons on most Web sites - it will draw relatively few responses, which brings into question the projectability of the feedback gathered (i.e., Are these responses representative of all visitors to the Web site?).
  • Keep it short. Even if you offer an incentive, it is best to keep the survey to roughly 10 questions, or five minutes of each visitor’s time. Beyond this point, you run the risk of boring people - and gathering responses which are not serious, thoughtful or complete.
  • Include open-ended questions. When asking people for their time, it’s important to give them the opportunity to speak their minds and make comments/requests in their own words. Asking only closed-ended, multiple-choice questions is shortsighted, because they will inevitably miss some unanticipated problems and opportunities. At the very least, each survey should include a question asking visitors for their ideas to improve the site (i.e., How can we make it better?).
  • Differentiate between first-time and repeat visitors. The feedback of repeat visitors is usually quite different from that of first-time visitors to a site - and far more relevant to marketers with respect to measuring the impact of the site (i.e., Has it influenced visitors’ attitudes and actions?). For this reason, repeat visitors should be identified (through a survey question) - and their responses should be analyzed separately. In addition, you may want to actively discourage first-timers from taking the survey before fully visiting/viewing the site.
  • Use site-based surveys appropriately. To use site-based surveys effectively, it is important to understand their limitations. To begin with, they are not the place to share confidential ideas, because your competitors are among the most frequent visitors to your site. Surveys are also not an ideal forum for guiding initial design or gaining an in-depth understanding of site visitors. Both of these objectives require more thorough discussion, and are thus better-suited to focus groups or personal interviews.

As these points suggest, site-based surveys are not the answer to every Web marketing issue, nor are they a replacement for traditional off-line research. If used properly, however, they can be a valuable tool for periodically taking the pulse of Web site visitors and gathering the feedback needed to assess and improve design. Ultimately, Web marketers who invest in this dialogue will have a better understanding of their site visitors and the insights needed to make their sites more compelling and effective.