When do you turn to the Web?

Editor’s note: Ron Weidemann is client service manager, and Chuck Sinks is vice president, at Answers Research, Solana Beach, Calif.

One of the first questions researchers ponder when planning to conduct a quantitative study is whether the data should be collected via the Web, phone or snail mail. Today, the most common modes of quantitative data collection are online or by phone. Data collection by mail is used less frequently due to dropping response rates and higher data entry errors, but that is a subject for a future article. When determining the best survey data collection mode, two key elements of your research need to be considered: target audience and survey content.

Who do you want to survey?

The optimal survey mode - Web or phone - will have the capability of reaching a projectable sample of your research target market. What is interesting about studies using the Web mode is the ongoing and rapid penetration of the Internet into business and consumer markets. Current estimates indicate that more than 80 percent of U.S. businesses and about 60 percent of U.S. households have some form of online capability. Among U.S. businesses and households with Internet access, broadband penetration is approaching 70 percent - making data-intensive survey designs (with images, video, etc.) more practical than in the past. In other countries, the volumes of broadband subscribers is even greater (the U.S. ranks about 15th worldwide) with Korea, Hong Kong, the Netherlands and Denmark leading the way.

Despite the Web’s penetration into business and households, there is still a perception that Web studies, especially using Web panels, cannot yet be projected to the general U.S. population. In the infancy of Web surveys this was true, but oh, how the times are changing.

Nowadays it’s the demographics/psychographics of the target respondent that indicate the mode researchers should utilize. For instance, approximately 8 percent of the U.S. population has already cut the cord of their landline telephone - that is they use cell phones and other communication devices instead of a landline. The percentage of U.S. “cord cutters” is growing at a rate of nearly 2 percent a year. Presently, those who have cut the cord are typically less affluent, most likely single and generally more liberal politically. Clearly, a landline phone study using random-digit dial is not practical when trying to reach these target respondents. Thus far, the cell phone-only group is not quite large enough to impact RDD survey projectability and accuracy to the general population - but that day is fast approaching.

Going forward, researchers need to be aware of the potential biases that could be created by not reaching the rapidly growing pool of respondents who no longer have a landline phone.

  • Subject characteristics

Frequently, the subject of the research - the product or service we are researching - will be a driver in determining which mode is best for collecting data from your target market. Searching for computer users? A projectable sample or panel can be easily found online from a number of excellent sources. Statistically, the vast majority of computer users are also online. While phone studies can certainly identify computer users, a Web survey is a much less expensive and a preferred alternative to reach these targeted respondents. Of course the contents, objectives and design of the survey must be compatible with an online mode - but computer users would be excellent candidates. Other natural examples of products where projectable online sample can be found are digital cameras, photo printers and flat-screen monitors.

  • People characteristics

Online research is best when your focus is more about reaching specific demographic groups of people rather than general product usage. Younger, less affluent singles are more likely to rely on e-mail, text messaging and cell phones to communicate. For example Jane, a college student, may not have a landline telephone. She relies on her cell phone, PDA and computer for her communication needs. Text messages and instant messaging are a way of life for keeping in touch. The most effective means of contacting Jane and others who are tech-savvy is through an e-mail survey invitation including a link and a password to an online survey.

Another good time to select a Web methodology is when you’re trying to reach extremely busy people. As you might imagine, we’ve seen higher response rates when busy people are given the opportunity to complete the survey on their own time. An IT system administrator is someone who is always busy, often on-call 24/7. Rarely (if ever!) does an IT professional have 20 minutes to sit on the phone and answer questions at the exact time an interviewer calls. However, IT pros are generally anxious to provide input on new technologies and are better than most of us at checking e-mail. You’ll increase your response rate and lower costs by giving busy people like this the option to complete the survey at their convenience - morning, noon or even midnight! Additionally, most online surveys have the capability to stop and restart the survey - in the event the respondent needs to break while in the middle of completing the survey.

What do you want to know?

Content also has a lot to do with determining the best mode for data collection. The Web is a good candidate for studies when 1) product concepts need a visual representation, 2) new product concepts need a detailed explanation, 3) a choice needs to be made between products with multiple attributes, and 4) there are so many options that it would be difficult for the respondent to remember them all without being able to see all options.

Being able to see words or pictures helps the respondent maintain focus. When listening to an interviewer over the phone, respondents have shorter attention spans - it’s difficult to recall a large number of items on a list. In fact the drop-off rate for telephone surveys over 20 minutes in length increases quickly as respondents fatigue. Online surveys enjoy 15-20 percent more questions than phone surveys before respondent drop-off rates begin to accelerate.

A picture can convey more about a product than words. If you were to describe every visual aspect of a product, the respondent would likely get bored. If you didn’t describe every visual aspect, you could be missing the one aspect that matters most to the customer. Take advantage of visuals if you can; the Web is the best medium for this. Of course the best visual aids are real products that can be shown in-person, but that is not always practical - financially or otherwise. There was a time when conducting an online survey with visual aids would have been difficult to do successfully because of bandwidth limitations and the inability to view images because of a lack of software (i.e., not having Adobe Acrobat, etc.). This is changing as broadband Internet connections are becoming more common. Software for video streaming and image projection has made it easier to view images online.

One scenario requiring visual aids is a home builder trying to decide between two layouts for houses. The home builder wants to identify which layout will sell more houses. It is too expensive for the builder to construct the models and have individuals tour the facilities. Instead the home builder can build a model, take photos or video and show it online. The respondents can then give their feedback on preferences.

There are other situations where a product concept has been conceived but a physical model has not been developed. Before making an investment in developing a physical model, interest in the concept needs to be assessed. This will require providing respondents a product description instead of a visual image. It is easier for people to understand a description in writing than over the phone. And, people can better visualize a new concept when they are reading it at their own pace than when someone else is explaining it to them at a pace that may not be appropriate for that individual.

With some conjoint studies, respondents need to identify whether a new product would be chosen over existing products. Photos of all the products can be shown with prices, brands and product descriptions. The respondent chooses which product they want. This helps to capture information that would not be included if only a product description were shown. The product description would not provide the aesthetics of each product and it could leave out one of the key features that respondents prefer.

Sometimes the choices respondents have are numerous. Being able to see the options visually is better than having them read over the phone. On average, people can only remember seven things at a time. Phone surveys require individuals to remember the list that was read to them while online surveys allow the respondent to look through the list and go back and forth.

Who and what

Identifying who you need to survey and what you want to know will indicate which methodology is best. Once the study parameters of who and what are established, it is easier to identify whether online surveys are most appropriate. Online surveys are most practical when reaching groups of people that are very busy and when the survey topic is related to computers or technology requiring the usage of computers. It is also preferred over the telephone for abstract concepts or topics that require the usage of visual aids or a detailed explanation.