Editor’s note: Linda Piekarski is vice president, database and research at Survey Sampling International, Shelton, Conn.
Marketing researchers have a variety of sampling methodologies to consider with each research project they initiate. Representation, cost, time in the field and efficiency are relevant issues that can vary mode-to-mode. Knowing the advantages and disadvantages of each sampling methodology helps researchers choose the most appropriate mode for a successful project.
A sample - a subset of a given population of interest - is used to make inferences about that population. Samples are used when a census or complete enumeration of the population is impractical. One potential source of sampling error is frame coverage error. If the frame from which a sample is selected is not representative of the population of interest, then the estimates may be biased, particularly if those groups omitted or under-represented are different in terms of unobserved characteristics that can’t be offset with sample balancing.
For example, the demographic biases associated with unlisted households, cell phone-only households and households without Internet access have been well-documented and their exclusion might lead to bias. Therefore, it is crucial to select an appropriate sampling frame and mode to match the research objective. A look at sampling modes typically used in the U.S. helps to illustrate this learning.
Face-to-face
Face-to-face sampling is based on an area probability sample and offers full coverage. Face-to-face sampling is conducted for many U.S. government studies and some private-sector studies. Expensive to implement, this mode is considered extremely representative as every household has a known non-zero probability of selection.
Address-based
Address-based sampling relies on deliverable addresses from the U.S. Postal Service’s Delivery Sequence File. Address-based files can be merged with consumer databases of names and/or addresses and telephone numbers, where available. This allows for multimode sampling, combining telephone with mail and even face-to-face. Less expensive to implement than face-to-face alone, address-based sampling provides excellent household coverage.
RDD telephone
Random-digit dialing (RDD) telephone samples historically covered 95 percent of U.S. households. Today, with approximately 16 percent of telephone households cell phone-only, list-assisted RDD sampling frames represent only 83 percent of all U.S. households. Despite the declining coverage, list-assisted RDD sampling remains the mode of choice for many research projects.
The inclusion of samples drawn from wireless telephone RDD frames will enhance the coverage of list-assisted frames. Although cell phone surveys present many challenges in the field for researchers, sampling from the combined frames will provide 97 percent coverage of U.S. households.
Directory-listed telephone
Directory-listed telephone samples are limited to landline households listed in a directory. On average, this mode covers 60 percent of U.S. households and will vary by geography.
Advantages of directory-listed samples include a higher working phone rate when compared to RDD samples. Directory-listed samples are considered for projects that require names and addresses on every record, geographic integrity or targeted demography.
Low-incidence targeted
Low-incidence targeted samples are ideal for connecting with hard-to-reach populations easily and efficiently. These databases contain primarily self-reported demographic and behavioral information, blended from multiple sources and available at both the household and individual levels.
Using this mode, the coverage of the household or the population is unknown. However, self-reported information provides the opportunity for researchers to locate that “needle in a haystack” lifestyle or demographic in a more efficient manner than screening an RDD sample.
Internet
According to the Pew Internet & American Life Project, 75 percent of adults use the Internet. Internet sampling offers researchers many advantages over other modes, such as increased speed from launch to data results, lower costs, and opportunities to incorporate audio/visual technologies. Internet samples also offer researchers a variety of demographic and lifestyle targeting options similar to those available in other modes. Disadvantages are that Internet sampling is only representative of people who are online or who can afford to be online. In some countries where Internet penetration is low, certain demographic groups are significantly underrepresented (e.g., older people in China).
Quite valuable
Combining methodologies can be quite valuable. In one instance, one of our clients was in the practice of acquiring mail-study participants by telephone, an effort the client found arduous. Online recruitment - accomplished in two days instead of a week by telephone - was more efficient and less costly. With panel quality of the utmost importance, the client did some comparisons of study data from surveys recruited online and those recruited by telephone, and found the results comparable.
A multimode approach can be beneficial in other research designs as well. These include online B2B studies recruited by telephone and a combination of RDD telephone, wireless RDD telephone and online sampling frames to ensure coverage of selected demographic groups.