Who makes up that respondent database?

Editor's note: Ron Sellers is president of Ellison Research, Phoenix.

Most focus group facilities have them, and they're generally taken for granted as much as the microphones in the ceiling or the bowls of M&Ms in the back room.

Respondent databases.

With all there is to worry about in a qualitative project, it can be tempting for the researcher to tell a facility who they want in the groups, and let the facility worry about where to find those people.

After holding three different sets of groups at two different facilities in Phoenix in a space of 18 months, and somehow having the same individual show up as a respondent for each project (on three totally different topics), I think it is definitely time to re-examine that approach.

Properly obtained and managed, respondent databases can solve a lot of the headaches for recruiters. Databases are made up of people who know who the facility is when they're called, and have already expressed their willingness to participate in groups if they qualify. Sure makes life easier.

I have some facilities I've used again and again, and I trust how they build and use their database. But what happens when client preference or a new market forces me to use a facility with which I have no experience?

First, I want to know how their database is built. Do they include people who call them and say, "I want to do focus groups"? If so, I'll pass. You know the person calling that facility has just opened the phone book and called all the other facilities in an attempt at some easy money once or twice a month. (We get those calls and e-mails all the time, even though we don't have a focus group facility.)

Second, I want to know how often people are contacted for any type of research. Not just how often they participate in focus groups, but how often they're contacted. The more those people are called, the more they have an idea of how to answer screening questions to make sure they're qualified for the recruit. In addition, some facilities will not use respondents more than every six months for focus groups, but will gladly contact them more frequently for in-depth interviews, taste tests, etc.

Third, I've learned to rely on the following test. When I contact a facility, I ask for a cost if they recruit from their database, and a cost if they recruit virgin respondents (i.e., from RDD sample, directories, or some other relatively comprehensive source).

What I've discovered has been fascinating, and fairly predictive of what my experience with the facility will be like. Some facility directors are nearly panicked by this request. "Recruiting from the general public is almost impossible!" they wail. These directors then demand much more recruiting time, charge far higher prices, and are already setting me up for eventual failure when they say, "We'll attempt this for you, but we don't know how well it's going to work if you don't use our database."

If a facility is really uncomfortable recruiting from the general public, it's not a facility I'll be using. Their discomfort shows they've become so fat and happy recruiting from an overused database of professional respondents that the thought of having to leave that comfort zone scares them.

Now, I fully expect higher costs and a slightly longer recruiting time when not using a database, if only due to the fact that initial refusals will be higher when dialing the general public than when dialing into a database. And it's fair to expect the need to over-recruit by an even greater number, because virgin respondents will be more likely to get cold feet at the last minute or decide it's not worth their time and not show up. But the cost-per-recruit shouldn't go from $75 to $130 because the database can't be used. It shouldn't take an extra week to recruit two groups. And we shouldn't have to recruit 16 for 10 to show.

Another tip-off is when a facility director will ask something like, "If we don't use our database, what can we possibly use to get names?" Facilities that have never heard of random-digit-dial sample, or that insist the sample quality is so poor that they'll never be able to get the project recruited, also generally will not get my business.

Still another hint there may be trouble afoot comes when the facility will ask us to reduce our past participation screening question from a time frame of 12 months to six months. This, too, suggests an overused database.

Certainly, there are times a database is the best way to go. If we have a ridiculously short period for recruiting, or the incidence is painfully low, a database definitely helps mitigate those problems. Some facilities maintain information on their database that will really help the recruit, such as what vehicle the individual owns or what her profession is.

There are also times when the budget is tight, and the client won't spring for an increased cost of even $10 per recruit, because it's "unnecessary." (I've found clients too often have the perspective of "I'm not worried where they come from - I just want to hear what they have to say.")

But in a normal-incidence recruit where we have sufficient time, I'll frequently insist on recruiting from the general public to get some fresh voices in the group. I'm tired of sitting down at the table, asking "How many of you have participated in focus groups before?" and having all 10 respondents raise their hands.

And every time that happens, I'm hoping the client behind the mirror is thinking, "Next time, we'll spend a little more to recruit virgin respondents rather than working off that database."