Rewriting the rules

Editor’s note: Debra Power is president of Moore Power Marketing, an Ann Arbor, Mich., research firm.

In recent years, recruitment and execution of focus groups has become a major challenge for marketing and research firms. First, recruiting participants has become more difficult as screening requirements have grown in complexity. As marketing becomes more segmented, researchers are forced to develop more intricate procedures for identifying and compartmentalizing participants. Second, potential recruits are continually bombarded by unwanted telephone solicitors, crippling one of the traditional recruitment methods. Market researchers are competing with telemarketers, caller identification technology, and wariness of telephone transactions.

Research firms are forced to develop new and innovative ways to reach out to the general population. This case study, although specific to the subject matter, should provide some guidance to overcoming these challenges. By discussing each aspect of the research project in turn we hope to provide insight and suggestions for surmounting the difficulties researchers are now facing. We will also present the new rules we had to develop to meet our specific challenges in the hope that they prepare other researchers for this new era of qualitative research.

Background

In early 2001, our firm was asked by a large academic medical center to recruit, moderate, and analyze a series of 12 focus groups on attitudes and perceptions towards colon and rectal (colorectal) cancer screening (testing) in Michigan. Colorectal cancer is the second-leading cause of cancer deaths among men and women in the U.S. The information gathered from the groups would be used to develop a Web site designed to convince the general public to get tested for the disease. These focus groups were held in locations which had a low rate of colorectal cancer testing, and which met population density requirements - Detroit (metropolitan), Saginaw (suburban), and Benton Harbor (semi-rural), all in the state of Michigan.

In order to participate, focus group recruits were required to be between the ages of 50-70 and be untested for colorectal cancer. A balance between high and low incomes was also required. Participants were further segmented by gender and ethnicity (Caucasian and African-American). The goal was to complete one focus group for each gender, ethnicity, and location.

As is often the case with health care research, adherence to budget restrictions during project execution was paramount because funding for the study was grant-based. In addition, the parameters of the grant (time frame, types of groups, locations, etc.) were pre-determined and we needed to comply with these as closely as possible.

At first glance, although segmented, the recruitment and execution of these groups seemed easily managed. Recruiting was to be done by telephone with a stratified random sample drawn from 1990 Census data, as 2000 Census information was not yet available. We knew this data would be somewhat outdated, but the study dictated that we implement as many random selection factors as possible. We would execute the project as any other: begin recruiting two weeks before each group, send confirmation letters, make follow-up calls, re-recruit if there were any cancellations, and hope that a sufficient number of participants attended. Unfortunately, this somewhat optimistic scenario was never realized. Throughout the project we encountered difficulties, but recruitment of participants was by far our greatest challenge.

The problems

A series of problems arose as we set the recruiting process into motion:

  • We were unable to fill quotas, even with recruitment starting three weeks ahead of time.
  • It took twice as long to recruit males (rates as poor as .25 completes per hour).
  • Dropouts and terminates during screening were unusually high.
  • The no-show rate was substantially higher than in our previous experience.
  • We were inhibited by various requirements of the grant itself.

In the face of these unprecedented difficulties, it became necessary to re-evaluate our entire recruitment procedure. This required us to come up with some new rules.

New Rule #1: Don’t be afraid to change the screener - and change it again until you have it right.

As mentioned earlier, our goal was to recruit individuals based upon age, ethnicity and testing for colorectal cancer, and a telephone screener was developed with this in mind. All calls to potential recruits were made by Robinson & Muenster Associates, Inc. (RMA) of Sioux Falls, S.D. According to RMA, our study was one of the most challenging recruiting projects in their firm’s history.

We surmised that one way to determine why recruiting was so difficult was to review when terminates and dropouts occurred in the screener. One question in particular - “Have you ever had a test for blood in the stool, sigmoidoscopy, colonoscopy, or double contrast barium enema?” - was a major source of terminates. Because so many respondents were disqualified at this point in the screener when they answered positively, the requirements were loosened by adding “Within the past 10 years have you had a...” to the start of the question.

RMA was also able to tell us that some dropouts took place in the opening paragraph as expected, but most occurred around the question about colorectal testing mentioned above. A certain number of respondents were squeamish, or unwilling to answer the question, and dropped out. To combat this difficulty we changed the screener again and moved the question to later in the screener (from #6 of nine questions, to #9). Interspersing demographic questions with health-related ones seemed to raise the comfort level of the respondent and reduced hang-ups.

Interestingly, the age, vocal tone, and gender of the interviewer had a significant impact on the number of completes. RMA noted that when calls were made by an older, gentle-sounding female, the response rate improved.

These changes to the screener resulted in decreased recruitment times and an increase in the number of potential participants. While this was a step in the right direction, we were still grappling with unfulfilled quotas and high no-show rates.

New Rule #2: Don’t be afraid to mix and match.

As recruiting progressed and we faced more adversity, we determined that methods in addition to telephone solicitation would be necessary. To try to reach out to the general population we began running newspaper ads in the regions where we were recruiting. The ads included an 800 number, and once a potential recruit called in they were taken through the screener. This method resulted in nearly 20 additional qualified participants.

After two groups were cancelled due to low turnout (only one and three individuals, respectively, out of 12 recruited), we decided to begin asking qualified recruits to recommend potential participants. Although this is not a preferred method, in the suburban and semi-rural locations it was effective, possibly due to lower overall population density and the corresponding familiarity of community members with one another. By mixing recruitment methods we saw a measurable increase in the number of recruited participants. Our new rules were making an appreciable impact but it still wasn’t enough to fill our quotas.

New Rule #3: Sometimes the simplest change can make a world of difference.

Some changes to the recruiting process were commonsensical. As mentioned above, early on, two groups had to be cancelled because of high no-show rates (92 percent and 75 percent). Thereafter we raised the number of recruits to 15 for 10 to show, instead of the usual 12 for 10 to show.

Also, about midway through the project, we started asking participants why they decided to attend the focus group session. Roughly 40 percent of participants cited the money, another 30 percent curiosity about the disease, 20 percent had a family history of cancer, and 10 percent had miscellaneous reasons for attending.

By asking this simple question we were able to gauge our target audience in relation to their receptiveness toward and interest in the topic. The popularity of remuneration led us to increase the incentive amount from $50 to $75, a change which proved to be very effective. The simple increase of $25 in the incentive was enough to pique the interest of many potential participants.

Another adjustment we made to our conventional focus group recruitment process was one of the easiest. When attendance at groups dropped, we felt that we needed to emphasize to potential participants that their input was very valuable. From the very start, recruited participants were sent a cordial confirmation letter and received a follow-up call in accordance with our usual policy. Halfway through the project we altered our procedures and increased the number of follow-up calls to two per participant. In the interest of maintaining a good relationship with the participant, this second follow-up call was often left on an answering machine. Again, the simple change of an additional telephone call resulted in lower no-show rates. But now we faced the greater challenge of addressing problems that originated with the project itself.

New Rule #4: Don’t allow the project requirements to become a hindrance.

One of the medical center requirements for this project was the distribution of a dense, three-page consent form to each participant which required their signature and acknowledgement. The form detailed the procedures of the study as well as the overall scope and purpose in a question-and-answer format. It also included contact information for the medical center for the participant’s future reference. Unfortunately, the form was a standard document also used for clinical trials. Thus, it included questions like: “What kind of harm can I experience in this study?” and “What will the investigators do if I get injured in the study?”

Originally, we sent this consent form along with the confirmation letter to all participants. We quickly learned after two or three groups that the form itself was inhibiting participation in the focus groups. Recruits would notice language on the form such as “harm,” “research subject,” “treatments,” and “procedures,” and think twice about coming to the focus group session. Also, they found the form itself confusing - the participant would read an objectionable word in the question section and did not read further for the corresponding answer. Both our firm and the medical center received several telephone calls from potential participants (and their family members) with concerns about the consent form. Several of those who did come to a group refused to participate unless the form was explained more fully.

We quickly decided to stop mailing out the form, and instead handed it out when participants arrived at the session. Although they were still uncomfortable with the form, and it took nearly 10 minutes to read, their fears were allayed by the ability to discuss the form with the moderator. By altering when we distributed the consent form we were able to simultaneously meet the grant requirements of the project and decrease our no-show rate of participation.

Degree of difficulty

At the end of two months, over 630 hours of phoning, and 11,560 contacts, a total of 10 focus groups were completed and analysis began. Throughout the project’s evolution several focus groups were cancelled and subsequently rescheduled, and one male group was conducted with combined ethnicities in order to gather as much qualitative data as possible.

At the genesis of the project we had no way to predict the degree of difficulty we would face with recruitment of participants. As we struggled along we rewrote the accepted procedural rules for recruiting to meet the goals of the project. At the same time, we had to make some sacrifices, but these were necessary in order to fulfill our objectives.

These new rules may not work for every research firm, or apply to every project. You may have to develop and refine your own rules or alter one of ours, but one of the most important things to remember is to keep your client informed. If you are having difficulties with any aspect of your project, remember the final rule (and it is a cardinal rule at our firm): keep the client happy. The best way to do that is to keep an open line of communication about all aspects of the research project.