Extracting maximum value

Editor's note: Jim Bryson is founder and CEO of 20|20 Research, Nashville, Tenn.

By definition, a by-product is something that is produced on the way to producing something else. Often it is waste; sometimes it is useful. Webster’s dictionary defines by-product as, “something produced in a usually industrial or biological process in addition to the principal product.”

As researchers, we do not think of qualitative recruiting as an “industrial or biological process” but there is no doubt that it is a process that we use to attain an end product, a qualified respondent. The qualified respondent is what matters and we pull out all the stops to find that survey participant.

A second definition of by-product is, “a secondary and sometimes unexpected or unintended result.” History is filled with examples of serendipitous discoveries of useful by-products that are sometimes more useful than the end product themselves. For instance:

  • Until the late 19th century, leftover yeast from your favorite brewer was simply hauled off and dumped as waste. Justus von Liebig discovered that it could be concentrated, bottled and salted to create a nutritious, low-cost food he called Marmite. It became a staple of troops in World Wars I and II and Marmite still bottles 24 million jars a year.
  • Coal tar is a by-product of the coal gasification process. In 1878, Constantin Fahlberg accidentally discovered its incredibly sweet taste, 300 times sweeter than sugar. He named his discovery saccharin, which is the primary ingredient in today’s Sweet‘N Low.
  • Spencer Silver was a chemist for 3M attempting to develop a better, tougher adhesive. But one of his discoveries – microspheres that could stick to most any surface, be removed without any residue and reapplied elsewhere – was anything but a stronger, hardy adhesive. It took a team member to point out that Silver’s break-through, when applied to small pieces of paper that could serve as bookmarks, was actually an entirely new way to communicate. And Post-it Notes were born.

By-products are everywhere because they create value in addition to the end product. Why would anyone throw away a by-product as waste when it has value that can be used in a new and different way on its own or to enhance the end product? But that is exactly what we do in qualitative research.

No regard

Most recruiting today is done with blinders on. We focus on the goal with no regard for the by-products produced along the way. When I started recruiting in 1986, each recruiter came to work with a yellow pad filled with names and phone numbers of potential respondents. This was their personal “panel” that they maintained and protected. At night, they took this yellow pad home hoping to add some friends or new acquaintances to the list of potential participants. These recruiters were on a mission to do one thing: identify and harvest qualified respondents.

Within a few years, we pried those lists out of their hands and built a common “index card” file that was kept at the office which all recruiters could access. A year or two after that we developed a simple database and our “panel” became digital. Soon, we began to use e-mail and more sophisticated database programs to target our potential respondents. Still, our focus remained simply to recruit more respondents more efficiently.

Never in our history did we consider the process of recruiting to be an asset. It was always a necessary cost as a means to an end.

Ignores the possibilities

To keep recruiting the same way we did 30 years ago ignores the possibilities that technology brings to the table. Think about it: We often screen hundreds of people to find a group of 12 to participate in a study. We invite that dozen lucky souls and then toss the screeners in the recycling bin or save the data on a server somewhere out in the ether.

But what about all that data gleaned from the thousands of respondents who complete our screeners but don’t qualify? Isn’t there something important to learn from them? Of course there is. It’s time to change how we think about recruiting. It’s no longer simply a task to meet an end; it is a process that can yield revealing information from start to finish and from every twist and turn along the way.

When researchers simply harvest the recruits and leave all the data lying in the field, they have relinquished valuable information that has been bought and paid for but discarded like unwanted junk mail. Harvesting from the recruiting process yields more information with virtually no increased cost. Doing otherwise seems a bit irresponsible.

After all, today’s recruiting is more challenging than ever with more and more demands to recruit from tightly segmented markets. It is not uncommon for a typical study to require respondents who represent far less than 1 percent of the market. To put that in perspective, a 1 percent incidence means 1,200 people must complete a recruiting screener for a study with a requirement to “recruit 12 to seat 8-10.”

To meet that demand, recruiting firms have incorporated panel software to run large databases of potential respondents with initial screening using e-mail with telephone follow-up. Recruiting firms are racing to increase panel size, improve systems and develop technology to assist in processing the massive amounts of data and people that must be accessed to fill a single study.

All of this effort and data accumulation is pointed toward a single goal: to produce a qualified respondent.

Do not recognize

The processes, panels and technology in use today produce data by-products that most researchers do not recognize, much less use. For example, imagine doing a survey of 1,200 people in your product category and throwing away the questionnaires from everyone who did not purchase your product at least five times a month. Do we really believe there is no value in learning from the hundreds of respondents who buy the product, say, four times a month?

To get started on making the most of the recruiting process, simply pay attention to the data gathered. Take a look at it to see what the data tells you.

Most online screeners use survey questionnaire software that stores responses in data files, just like any other survey. The data is structured and available to be analyzed using survey software. The only difference is that records are “incomplete” because of the screeners’ “terminates” during screening for qualified respondents. This does not mean the data is unusable; it simply means the researcher will have to consider different sample sizes for different questions/variables in the data set. The data gathered at the beginning of the questionnaire will be very robust with many completed interviews. The data gathered toward the end will be much less so.

Technology and panel data help in other ways as well. Most panels have demographic data on their panelists to assist with quickly locating the qualified respondents. Usually, panels have at least five demographic variables available: age, income, gender, household income and education. Appending these variables to the data set can make it much more robust, even for those questions with smaller sample sizes because they appear near the end of the screener.

These data are by-products because any findings add supplemental value to the primary product of finding a qualified respondent for the specific research project and marketing question at hand.

In the example above, we hypothesized a project in which qualified respondents had to purchase a product five times a month. If these “heavy users” were a 1 percent incidence, what do you think we might learn from the screener data about users who purchased once a month or three times a month? We might learn that they are older, younger or less educated. We might discover that they are heavy users of a competitive brand. What would be the implications if we discovered that they would like to buy the brand more but the brand is simply inconvenient for them?

In this example study of heavy users, the potential findings related to the lighter users are irrelevant. They have nothing to do with the study’s purpose of conducting research among heavy users. The researcher simply has no reason to ask the questions or to investigate the answers. So no one ever looks at the data. No ancillary learning takes place. Removing the blinders allows researchers to expand scope and learn more about the product, the category and users using data already being collected and paid for.

Crucial to the success

Not only are data by-products useful at the end of a study, they are often crucial to the success of the recruiting project itself.

Researchers usually cannot predict the incidence of a project. It is simply unknown. They send a list of specifications describing the potentially qualified respondent. The recruiting firm is expected to guess at the incidence and provide a “firm quote.” The result is often disappointment on all sides. First, the recruiting firm is frequently disappointed because qualified respondents fitting all the specifications and quotas were much more difficult than expected. Who knew finding men who chew Juicy Fruit five times a week would be so hard?

The researcher-client becomes frustrated as well. After all, they need those Juicy Fruit gum chewers for the focus group coming up in three days. The date is set. Travel has been confirmed. The research is on a fast track. What should they do? Invariably, they begin making compromises and accepting “holds” at the last minute. Do any of the men on hold chew Juicy Fruit only four times a week? How about three times? Two?

Suddenly, this project is no longer the “heavy user” research originally anticipated. The research is forced to compromise. Why? Because the incidence was not known up front. Client, researcher and recruiting firm were all guessing at how difficult this project would be. The result is that the recruiting firm started putting people who were anywhere close to qualified “on hold” while they churned away at the recruit, trying desperately to find men who chew Juicy Fruit. As the days go by, everyone’s anxiety levels rise. Finally, they reach a point where compromise is unavoidable; they start accepting “holds.” No one is happy.

Recruiting technology and the data by-products can help us to eliminate the guesswork and much of the anxiety though dynamic recruiting.
Dynamic recruiting is a simple process that requires a partnership between researcher and recruiter. Once the screener is developed, the recruiting firm and the researcher agree on a process similar to the one below.

Blitz screening: Use the initial screener to screen several hundred respondents within the first two days of recruiting.

Pause and evaluate: Pause screening for a day to review the data generated as a by-product of the actual recruiting. This data provides objective evidence related to the study incidence. With this data we can accurately predict how many people will need to be screened to complete the study. The researcher and the recruiter can confidently agree on any adjustments to the screener and/or the bid.

Complete screening: Screening should be completed on time, at an agreed upon cost and with a minimum of holds.

Technology and panels dramatically speed the screening process so that the blitz screening as described above is possible. With data from a few hundred respondents, decisions can be made early in the process that virtually eliminate last-minute holds and the research compromises that come with them. If adjustments must be made, they are made early when time is available to adjust and rethink the implications. After the pause and evaluation day, screening is much more predictable and successful.

The advantages to dynamic recruiting are significant:

  • Anxiety reduction: Decisions are made using data and not simply guesstimates. The data is analyzed early so decisions are made in time to make any necessary adjustments in screening, expectations and the research it-self.
  • Better research: Since adjustments are made early, they can be made with the entire research project in mind. The early data analysis also adds learning to the project that can be incorporated into the research design. By nature, last-minute adjustments are detrimental to the entire research process.
  • More accurate bidding: Recruiters are not required to bear all the risk of the recruit, nor do they have to dedicate days of project management to an unsuccessful recruit. Therefore, they can bid it accurately, knowing that their researcher partner will work with them to adjust the screener or the bid as necessary during the “pause and evaluate” day.
  • Lower costs: Today, easy recruiting projects are bid artificially high because difficult research projects are bid artificially low. Knowing that they will be able to bid each project according to its difficulty or incidence gives recruiting firms the leeway to lower their prices on recruiting.

Adjust our processes

We’ve come a long way, baby. From names on yellow pads to sophisticated mega-panels, it’s a new day in qualitative recruiting. As researchers and recruiters we need to adjust our processes to the information available to us. It is no longer acceptable to don our blinders in a hyper-focused effort to simply find qualified respondents. Technology and the data it produces make dynamic recruiting possible. It’s time for researchers and recruiters to form true partnerships using this recruiting method so that risks are shared, costs are lowered, expectations are met and research is improved.