Editor’s note: James H. Nelems is CEO of the Marketing Workshop Inc., a Norcross, Ga., research firm.

With the increased use of the Internet as a means for data collection by market researchers, greater attention is being given to differences between Internet and telephone samples. Several factors can contribute to differences between the two - lower adoption rates of the Internet among lower-income households, the growth of cell phones, which are not accessed by RDD telephone procedures, and the fact that Internet surveys are self-administered at the convenience and leisure of the respondent.

Since 2005, we have conducted an A&U tracking study in 12 DMAs for a national CPG client. In order to ascertain the impact of switching the data collection methodology, parallel surveys were conducted in each market, completing 200 interviews by Internet using an established household panel, and 100 by telephone with a randomly-selected sample.

While there was some variation across markets, the consolidated samples show Internet respondents tend to be younger with somewhat lower household incomes. One can always weight the data, but in our experience weighting rarely makes enough difference to matter. Internet versus phone is more than comparing demographics - that is the easy part.

There were similarities across markets in terms of brand awareness, brand trial and stores shopped. The differences between the Internet and telephone samples on unaided brand awareness, brands purchased recently, brand share of last 10 purchases and primary store shopped tend to be minor, while the Internet sample yields much higher levels of claimed brand awareness, brands ever tried and stores shopped occasionally.

One explanation for these findings is that the list of valid responses for certain questions - top-of-mind awareness, brands last purchased, brand share and primary store shopped - is a single answer regardless of data collection method. For other, broader questions (claimed awareness, brands ever tried, stores shopped occasionally) the respondent’s memory is stretched, the list of legitimate responses is longer and Internet respondents, with the time and visual list of names afforded by a self-administered survey, can provide a more thorough response.

In conclusion, be sure to consider the following points when choosing your methodology:

1. Both telephone and Internet-based methods of data collection may not be representative of your target audience - phone because the younger age groups are under-represented with the prevalence of cell phones as well as not being as responsive to surveys as their older counterparts. 

2. Never switch from one method of data collection to another without a parallel study to determine which differences may be due to methodology and which are due to changes in the marketplace. By splitting the sample (half phone and half Internet) the phone portion provides current tracking data against which to compare past efforts while the Internet portion provides a base against which to measure future Internet findings.

 3. Consider what measures are most needed for your study. If unaided measures are most important, it may be that the phone methodology works best. However, if aided measures such as brand ratings are most important, then the Internet may be the preferable methodology for speed, timing and access.

 4. Consider the purpose of the research in choosing between phone and Internet. One example is that a self-administered Internet survey may be more objective and provide better answers if there is some degree of hesitancy of verbalizing a fear or comment on a sensitive subject (such as sex or gun control or employee attitudes). Also, there may well be situations where expanded, thoughtful answers are desired.

5. Whether the methodology is phone or Internet, over time the measures will move in the same direction. However, absolute changes should not be compared across methodologies.