One true methodology?

Editor’s note: Dr. Murray Simon is president of D/R/S HealthCare Consultants, a Charlotte, N.C., research firm.

Recently a client chose to eliminate the triad segment of a proposed qualitative primary research project that was to start with triads and end with individual depth interviews (IDIs). Her reasoning was that it represented a mix of methodologies with a potential to generate confusing results.

It was a first project with this client and I thought it best not to initiate a debate (or pick a fight). The IDIs went well, but I firmly believe the results demonstrated a distinct deficit in the area of idea-building elements that might have been gained through the dynamics of group interaction. We went in with a series of suppositions that were verified or eliminated but we probably missed generating additional ideas to be tested and evaluated.

The more I thought about it, the more I found myself focusing on the phrase “qualitative market research methodologies.” Exactly what does that mean or convey? In my view, the term “methodologies” is inaccurate and misleading. As I will set forth below, I believe there is only one true qualitative market research methodology.

Triads, dyads, focus groups, IDIs and telephone depth interviews (TDIs) are not truly methodologies; they are the venues that allow us, as moderators, to make use of the one true qualitative market research methodology. Obviously these venues influence what we can and cannot do in any particular project - you cannot pass around proprietary advertising concepts for evaluation in the TDI setting nor can you break an IDI up into competing ideation teams. Certainly different venues produce varying dynamics (as does the quality of the recruitment) but regardless of the specific venue or caliber of the recruitment, the one true qualitative market research methodology is the major determinant of a project’s success or failure.

Go back through your files and look at various discussion guides. I suspect you will find the equivalent of the following:

- introduction and warm-up;

- current buying (or prescribing) activities;

- usage patterns and perceptions of unmet needs;

- presentation of a Product X profile, concepts or prototype followed by positives/negatives and likelihood to buy, prescribe or use.

With the obvious exception of specific prompts such as moderator passes the prototype around the table or have respondent put the concepts in rank order from most to least compelling, the discussion guides remain very much the same. And yet, some projects are more successful than others.

Have you ever come out of an IDI with your adrenaline pumping because of the masterful way you managed an initially difficult respondent, only to follow with an interview that put everyone in the viewing room to sleep? Have you ever moderated a focus group that performed like a well-rehearsed professional choir, only to follow with one that came across as the choir from hell? I know what you’re thinking: It was probably a bad recruit. Perhaps it was, but could it have been salvaged by better use of the one true qualitative market research methodology?

All right, let’s cut to the chase. What is this methodology? Just as any successful singer relies on preparation and execution, it includes the following:

1. The moderator must quickly establish rapport with the respondent(s).

2. Questions must be asked in a clear, concise and unbiased manner.

3. Respondents must be motivated or prompted to give comprehensive and candid answers.

4. The moderator must know when to hold ’em and when to fold ’em.

Let’s take a closer look at each of these components.

1. Establishing rapport

My work is exclusively within health care. Approximately 85 percent of my respondents are physicians and other health care professionals; the remainder is patients. These professionals are well-educated, highly trained and often suspicious that the moderator may have a (sales or decision-influencing) hidden agenda. When I go out to greet them and bring them back, as I always do with IDIs, they’re wondering how naïve will my questions be, how badly am I going to mispronounce the multisyllabic medical terminology and what do I really want from them. The first impression helps set the tone for the interview and, hopefully, diffuses their implicit concerns. It should be tailored to fit your personality, comfort zone and level of experience. After years of trial and error, my initial approach has evolved into:

“Dr. Smith, my name is Dr. Simon. I’ve come all the way from North Carolina to interview you. [pause] We’ve heard of you there. [longer pause] Some of it was positive.”

It’s safe to assume some of you would be quite uncomfortable with such an unconventional tongue-in-cheek approach (and many of you probably do not live in North Carolina). While it is not 100-percent effective, it does work well for me most of the time! It gets a smile from most respondents that is frequently followed by a question or comment about life in North Carolina. I believe it often helps in disarming those “hidden agenda” concerns among physicians. If there is no reaction or a negative one, at least I have an idea of the type of personality I will be dealing with for the next hour or so and can adjust my approach accordingly.

Patients, on the other hand, obviously require a different approach. While no two people are alike, there are two general patient respondent types to be found:

  • Those who have made an ongoing study of their disease, know a lot about it and hope to learn more from you (while, perhaps, at the same time educating you as to their information and treatment needs).
  • Those who are obviously not students of their disease but hope to learn about a new “magic bullet” during the interview.

My typical initial approach here is as Murray Simon, a specialist in health care market research who, as of now, knows very little about the ramifications of the disease being studied. I challenge them to educate me and hold nothing back, no matter how personal.

I believe it is critical to develop a first approach that goes beyond the standard “My name is Mary Smith. Did you have to come far to get here?” This person is about to spend 45 to 60 minutes in close interaction with you: don’t be afraid to open up a bit and express your personality. Try to break the ice first before you jump in.

2. Questions should be asked in a clear, concise and unbiased manner

Remember, success is highly dependent on preparation and execution (coupled with a bit of experience). If you’re working with physicians and mangle the technical language or demonstrate a lack of preparation, you’re likely to get dumbed-down responses and a respondent who is impatiently looking at his/her watch. Quite obviously, you’re going to come out of a study knowing a lot more than when you went in, but you have to start off with a demonstrable foundation of comprehension in order to hold the physician’s attention and keep him/her interested in the interview process (thank goodness for Google!).

This matter of unbiased questioning is an all too common issue. We often deal with clients who have a strong vested interest in the results of their studies (and perhaps some hidden agendas we’re not aware of). How often have you been asked/told to insert language into the discussion guide that sounds more like a sales pitch than a quest for information? “If Product X costs 10 cents per tablet, works 100 percent of the time and has no side effects, what is the likelihood, doctor, that you would prescribe it?”

Of course it is your responsibility to point these “flaws” out to the client and suggest alternate ways to frame questions. If, however, they insist that you “stick to the script” (and a client once actually used those very words with me), at least you know you tried.

3. Motivate respondents to give comprehensive and candid answers

Physicians generally participate in market research studies for one or more of the following reasons:

  • They hope to learn something about a new product or a new indication.
  • It satisfies their egos (as the scientist, the professor).
  • It may afford insight into what their colleagues around the country are doing.
  • It provides a welcome contribution to their kids’ college funds.

Discovering and playing off of their reasons for participating can be very useful in motivating more comprehensive and candid answers. Over time you develop a sense for what is motivating their participation. If the study includes a “Product X” evaluation, be sure to mention it a time or two early in the discussion - “In a while I’m going to give you a description of a potential new product to evaluate and I will be most anxious to get your thoughts on it” or “I’m looking forward to hearing what you have to say about a potential new treatment option for this disease.” This allows them a few minutes to get into their professorial, scientific mode.

Respondents subliminally want to know how they’re doing. I attend a one-hour indoor cycling class at my local YMCA three or four mornings a week. It starts at 5:45 a.m. and in order to motivate us at that ungodly hour, the instructor/coaches frequently compliment the class on how well it’s doing and encourage us to do more. In that same vein, I always try to drop in a compliment early in the interview. My favorite is, “I really appreciate how complete and concise your answers are. It’s very helpful; keep it up.”

Of course, motivation sometimes requires a confrontational approach, especially within the focus group setting: “Time out, ladies and gentlemen. Your attention is drifting and I need to remind you this is a paid work session not a social hour. Let’s get back to the task at hand, please.”

4. Know when to hold ’em and when to fold ’em

You have to sense when a question has been adequately answered and that it’s time to move on. And in that regard, sometimes you have to get tough with clients, especially those who are constantly rewriting questions and scripted lead-ins throughout the interviews (also known as rebuilding the plane while you’re in the air). Such activity is sometimes related to client inexperience and insecurity, but more often than not it is based on underlying hidden agendas, i.e., the study is intended to prove (often incorrect) assumptions rather than generate new and previously unknown information. This problem is often complicated when various members of the research team approach you individually and privately with their “unique ideas” for properly framing the questions that bother them. It’s time to take the most senior person aside and respectfully suggest he/she appoint a backroom spokesperson to field and filter suggestions from the group.

I once had a senior VP sitting in the back room among a group of mid-level marketing and brand management types. They were working hard to demonstrate their individual degrees of comprehension, which consisted mainly of debating the best way to reframe a particular segment of the discussion guide that was not producing the results they thought it should. After an hour or two of unproductive trial and error he finally spoke up and said something I find myself often quoting: “Ladies and gentlemen, sometimes ‘I don’t know’ is the answer.”

Obvious effect

So there you have it. Granted, clients and recruited respondents do vary and have an obvious effect on the research outcomes. But success in qualitative market research depends on how well you have mastered and make use of the one true qualitative research methodology.

Looking at it another way, it’s the singer, not the song. Whether Plácido Domingo is singing on the stage at La Scala, with the Three Tenors in Yankee Stadium or in his living room for a small group of friends, if he is in good voice (the true methodology) he will succeed; if he is not in good voice, the song (and the venue) will not matter.

(Author’s note: This article was intended to be deliberately confrontational. As such, I hope it generates some level of disagreement and perhaps an intense desire to debate the issues raised.)