Editor’s note: René N. Bos is research director at NIPO, The Market Research Institute, Amsterdam, the Netherlands.

In early 1999, after two years of development, our firm, NIPO, launched a new methodology of data collection for market and opinion research: NIPO CAPI@HOME. It is a methodology which uses a database of 10,000 Dutch households, or some 25,000 individuals, all of which have a PC at home and have agreed to participate in research projects using their home PC. Each PC has been equipped with our proprietary data collection software (ODIN) and all households have been fully briefed and trained on the use of this application, which involves filling out questionnaires that are sent to their homes by modem or via the Internet.

Since reaching operational status earlier this year, we have already completed some 250 ad-hoc market research projects and opinion polls and some 40,000 omnibus interviews by means of our NIPO CAPI@HOME.

This article’s objective is to explain why we are convinced that this methodology represents a new era in data collection. First, we will explain the background and motives for developing the new method. Second, we will get into more detail how it works and which applications we have found it particularly suitable for. Third, we will deal with a number of important issues and conditions for success, like respondent retention, and the need for incentives. We will close with a look at what the future may hold.

Combination of circumstances

The development of CAPI@HOME has been made possible by the combination of circumstances in market research that are forcing us to rethink conventional ways of data collection, and socio-technical changes.

The factors that forced us to rethink current methodologies include:

  • Decline in response-rates. At NIPO, as elsewhere in many parts of the world, we are experiencing declining response rates for both telephone research (CATI) and face-to-face interviewing (CAPI). Despite all imaginable technical and non-technical measures, we are facing a downward trend that seems difficult to stop. This has led to a situation where a 30 percent response rate (without recontacting) in both CATI and CAPI research is no longer exceptionally low. Of course, this is a major cause for concern.

First, because response rates of this magnitude and lower raise questions about the representativeness of our studies. Who are the people who are still willing to respond in our surveys? Are they different from those who refuse to do so? The lower the response rates, the more important these questions become. Ultimately, it would result in the need for heavy reweighting of virtually every study, and even the situation where the reach of certain target groups such as higher-educated singles or dual income couples has become so minimal that it would affect the validity of the research to a methodologically unacceptable point.

Second, because it threatens to affect the speed with which field work can be set up and carried out for our customers, and third, because it negatively affects the costs of research.

  • The interviewer factor. Another element that forces us to rethink current methodologies is the interviewer factor. Good interviewers nowadays are difficult to find and recruit, and even harder to retain. One cause for this is the boom in call centers in many urban areas. Call centers for research, direct marketing, and company help desks have become very popular, and they are all fishing in the same pond for interviewers or agents: usually higher-educated young people, often university students with good communication skills. In addition, the booming Dutch economy has made part-time jobs easily available anywhere, adding to the shortage of qualified interviewers.

These people are not motivated by pay alone. They also want to enjoy their work as an interviewer and like to have pleasant and intelligent conversations with their respondents. It doesn’t take much imagination to understand their frustration if they spend a lot of their time keying in reasons for non-response instead of conducting proper interviews. This adds to the difficulty of retaining these valuable people. Not to mention the understandable reluctance to go door-to-door (often in the rain) to conduct face-to-face interviews.

In addition, in the Netherlands, as in other countries, the authorities want to force research agencies to pay social security benefits on top of interviewer wages, which would add to the costs of field work, as do the costs of recruitment, which becomes more and more difficult.

But there are also other sides to the interviewer factor. Interviewers can introduce interviewer bias, despite countermeasures like briefing, training and instruction. Besides, it is not always absolutely necessary from a methodological point of view to have an interviewer conduct the interview.

  • Privacy legislation. A third factor that forces us to rethink current methodologies is privacy legislation. As a result of the boom in all kinds of telemarketing, lawmakers in the Netherlands and elsewhere in Europe are tightening privacy laws and regulations. These laws will make it more and more difficult to approach potential respondents without their prior written consent or at least without pre-notification letters. The database (blacklist) of people who do not wish to be approached for any kind of marketing or research activity, effectively also prohibiting legitimate research agencies from contacting them, is growing daily. This factor also negatively affects representativeness, as well as costs.

The factors that enabled the development of CAPI@HOME:

  • Dutch households become wired rapidly. When we started our computer-assisted Telepanel 15 years ago (with PCs provided to the households by us), computer penetration in Dutch households was very low. Now, it approaches 60 percent. And though the number of Dutch households that are connected to the Internet is still lagging at just below 20 percent, there is an unmistakable trend towards on-line penetration to levels that are now considered normal for the telephone: virtually 100 percent for businesses and households alike.
  • Increased computer literacy. With rising computer and Internet penetration, we also see a growing familiarity with computers and increasing computer literacy. At first, many segments of the audience were reluctant to touch a PC, let alone to purchase and to use one. We now see that elderly and lower-educated segments of the market are also getting used to the phenomenon and - though at a lower rate - getting wired as well. We are convinced that in terms of representativeness as well as familiarity and acceptance, computers and the Internet are and will be methodologically useful tools for research - if used correctly.
  • The reasons for non-response hold the key. Let’s turn back to the reasons for non-response. Over the years, we have studied the development of the reasons for non-response for all methods of data collection. Refusals have shown a significant increase, amongst our target groups - consumers and professionals - for a variety of reasons.

    One important reason is that there is a growing dislike of cold-call sales techniques, and with this, a growing suspicion towards every contact attempt - even when it is for genuine market research or opinion polling. So unless the respondent can be convinced immediately that you are not trying to sell anything, you are increasingly likely to end up with a refusal.

A second important reason for non-response is related to the fact that interviewers (telephone and face-to-face alike) only have a limited time window during which it is appropriate to approach households or businesses for interviewing. Time reasons account for almost half of all non-response. In most cases, consumers ideally should be approached at night, to try to ensure representativeness of the sample, but not later than 9:30 p.m. And we wouldn’t dare approach them on Sundays. Face-to-face interviewers encounter the same problems, aggravated by the growing reluctance people have to answer their doors at night, especially in urban areas. Interviewers themselves are also not too keen to walk around certain areas at night carrying a $2,000 laptop computer.

A recent internal study showed that in fact many people do have an inherent willingness to participate in research, but under certain conditions. They want to know that it is genuine research, not a sales pitch. They want to know the study is being conducted by a legitimate research agency. They would like to be pre-notified that they may be approached for research. And most importantly, they would like to decide themselves when they answer the questions. Using in-home CAPI, that can be at 9 a.m. on Sunday or even in the middle of the night!

It is very difficult with conventional methods of data collection to overcome the challenges to respondents’ willingness to participate. But CAPI@HOME can. We avoid the element of surprise; respondents know the research is legitimate; and they have the option to answer the questions whenever they like.

The development of CAPI@HOME

Some five years ago we started to continuously register household computer and Internet penetration in all our forms of data collection. The initial purpose was to monitor the development of this penetration and to conduct ad-hoc research on specific target groups.

Two years ago we developed the idea to putting this giant database to work for us. Experiments were conducted to evaluate willingness of the households and their members to participate, and the conditions under which they would do so. We experimented in recruitment and many technical issues like communication and compatibility. And so we recruited the participants from our own proprietary database.

Obviously, our experience with our Telepanel proved very helpful in the development. But again, CAPI@HOME is not a giant Telepanel. It is a database. A large amount of data has been gathered already from the 10,000 households (and their members) that have agreed to participate, including size and composition of household, age, gender and education of all members of each household, social class, life cycle, and a variety of behavior, penetration and incidence variables. All of this data can be used for selection of samples for research. Obviously, not all 10,000 households need to approached for every study.

Questionnaires are programmed at our firm and distributed to the selected sample by modem or via e-mail. Usually, we will send a pre-notification letter either by e-mail or conventional mail. This letter, as does the introduction to the questionnaire, clarifies for whom within the household the questionnaire is intended. This can be either a specific person or the person responsible for the household shopping, for instance. The introduction also clarifies the deadline and other essential information needed for completing the questionnaire. After the questionnaire has been filled out, the respondent establishes contact with the NIPO server in Amsterdam and sends the data back either by modem, by e-mail or via the Web, depending on the technical set-up.

This approach offers several advantages. First of all, the quality of data is high because respondents needn’t hurry to finish the interview in five or 10 minutes. They can take their time, even look up items such as bank statements if required. They can even temporarily break off the interview for lunch, or for two days, after which they can pick up where they left off. In the absence of an interviewer, there is a help desk the respondent can turn to for project-specific questions or technical problems they may encounter. Obviously, there is also an on-line help-screen with instructions for each questions, as well as an FAQ sheet.

Second, the response rates are very high. Usually in the area of 85 percent - within one week, with very often 40-50 percent within one day! Re-contacts, by e-mail, telephone or mail, are possible but hardly necessary. With these high response rates, we have also solved the issues of selective non-response which are typically found among one-person households, and working people, especially working couples, for instance. These high response rates contribute to speed, representativeness, and reduced cost per interview.

A third advantage is the elimination of potential interviewer bias, as discussed above. And with the written consent we have from all participants, we have safeguarded ourselves against any current or future privacy law.

Applications

Obviously, the database can be utilized for studies that require large samples. But a more frequent requirement is to conduct studies among specific target groups - such as owners of certain makes of automobiles, or middle-aged women, people who like gardening or have been on a vacation to a faraway continent. The reduction in costs and time as a result of the reduced need of screening is obvious. This method has already enabled us to conduct studies hitherto prohibitively expensive and time-consuming.

Conditions for success

Of course, there are a number of critical factors that determine success or failure. The challenge is to find the ideal frequency of contacts (again, this is not a panel approach!). On the one hand respondents should not be approached so often that we run the risk of fatiguing them or turning them into "professional"respondents for whom participation becomes a routine. But we must contact them frequently enough to keep them active participants in the database. Currently we aim at one contact (interview or set of interviews) per household each four weeks, but this varies, obviously, as does the average duration of each interview, which we want to limit to approximately 25-30 minutes.

Essential in the continued participation is the convenience for the participants of data communication and the user-friendliness of the data collection software.

Incentives are necessary to retain respondents. Currently, we employ a mixture of incentives. These can be gift vouchers for department stores in the respondents’ area, a small gift to a choice of charities and, something we have recently added, airmiles. Typically we award one gift voucher (of approximately $5) for each interview. But of course we also try to build rapport with the respondents, to ensure their commitment, using things like a newsletter with highlights from and background on research projects, and newspaper clippings of articles based on the research.

Implications for questionnaire design

Obviously, the fact that the interviewer has been eliminated from the process and the respondents themselves read the questionnaires from their screens has implications for questionnaire design. The most important implication concerns questions that require unaided answering, such as spontaneous brand and advertising awareness. In a face-to-face situation, questions such as these can be pre-coded for convenience of the interviewer and for ease of further processing. Using in-home CAPI, these questions cannot be accompanied by pre-coded answers; respondents have to type in the answers themselves. Another issue in questionnaire design is preventing the possibility scrolling back (where necessary), to avoid misguided correcting or adding of answers to previous questions. (If a respondent has made a mistake and wishes to correct it, there is always the option to mention that in a remark at the end of the questionnaire.)

Representativeness of results

An important issue is that of representativeness of results. Our database of 10,000 Dutch households cannot yet be considered fully representative of all Dutch households because non-PC-owning households have been excluded from the sample. But, as we argued before, this is a temporary situation and bound to improve soon.

But the size of the database does give researchers the option to select samples that are representative on a number of major variables such as region, size and composition of household, age, gender and social class. And if necessary, reweighting of results can also be applied.

The key issue, however, is the question whether the exclusion of non-PC owning households introduces a bias in the database. We still see some differences between PC-owning and non-PC-owning households, such as penetration figures for high-tech appliances, and also in attitudes and behavior with respect to purchase and usage of high-tech equipment in the household. This is a possible but temporary limitation of the database. But on the other hand, it enables easy recruitment of owners of many brands of PCs or electronic organizers.

However, in many instances, differences in characteristics, attitudes, behavior, penetration and incidences between an ordinary sample of Dutch households and their members and that of a CAPI@HOME sample that has been controlled for the variables mentioned above are minimal or non-significant. Graphs 1, 2 and 3 illustrate this.

The graphs refer to an internal study we conducted last year to compare the results of a conventional cross-sectional study among adults 16 years and older by means of CAPI with that of a sample from our CAPI@HOME database, drawn to be nationally representative on variables such as region, social class, age and gender.

In Graph 1 we see that the level of worry about a number of contemporary social issues is virtually the same in both samples. The same applies to the purchase level of ordinary articles such as shampoo and deodorant, as illustrated by Graph 3. Graph 2 shows the similarity of both samples in everyday behavior such as TV viewing and newspaper readership.

However, it is important to note that national or cross-sectional representativeness is not always a study’s requirement. Very often, research projects have to be conducted among specific target groups, where cross-sectional representativeness is not a key issue. For studies such as these, CAPI@HOME is an effective methodology, as the database gives researchers many variables that can be used for respondent selection and recruitment.

Future developments

CAPI@HOME is currently limited to the Netherlands, but in principle this methodology is without boundaries, so the application is possible in a multi-country setting as well. Also, the method is feasible in a business-to-business setting, CAPI@WORK, which we are developing.

Of course, more conventional methodologies such as CATI and face-to-face CAPI research will always remain necessary. So we are developing a new approach for CATI research (CATI@HOME) with improved attractiveness to potential interviewers (because they can work from home) combined with all other advantages of modern CATI applications such as remote listening in, field work and quality control.

Research is a wonderful and challenging profession and the opportunities for future development are endless. They always have been and definitely will continue to be as the personal computer keeps revolutionizing our personal and business lives.