Editor’s note: Michael Feehan is CEO, John Hartman is vice president and Denise Wong is senior associate of Observant LLC, a Waltham, Mass., research firm. Linda Lawrie is director of market analytics at Pfizer Inc., New York. This article appeared in the December 8, 2009, edition of Quirk's e-newsletter.
We’ve all heard the refrain, “It’s all about the (insert a preferred attribute)!” when end users discuss the features of a research study that communicate quality and excellence and lead to their ultimate satisfaction. For buyers of health and pharmaceutical market research studies, key factors range from the rapport the moderator establishes with respondents to the graphical clarity of the final report to receiving actionable results. Faced with myriad possibilities, market research suppliers must find out which elements are most critical to their clients and adjust their modes of operation, deliverables and consulting styles accordingly.
To determine if it really is the moderator or some other characteristic that drives satisfaction in the health care and pharmaceutical industries, we surveyed market research buyers and modeled the relative predictive utility of several project/vendor attributes. Based on the results, if we were asked to fill in the blank, it would go something like this: “It’s all about anticipating and meeting the client’s needs with a clear report!”
Research approach
Respondents
An online survey was completed by 42 U.S.-based market research buyers. Respondents were screened to be research purchasers in U.S.-based companies that had commissioned at least one primary research study in the health and well-being category from a market research supplier company in the past 12 months. At least one of their studies had to be U.S. market-focused. Respondents who conducted only ex-U.S. research were not included in the survey, as the factors driving satisfaction of international research (e.g., simultaneous translation, transcript quality, travel arrangements, etc.) were beyond the scope of this study. Potential respondents were identified from the membership list of the Pharmaceutical and Business Intelligence Research Group, augmented by commercially-available sample sources.
Respondents predominantly conducted pharmaceutical projects (81 percent), managed-care projects (29 percent) and hospital/clinic projects (24 percent). The majority were highly involved in most or all stages of the research process (93 percent), with the remainder only being involved in select stages or at a high level.
Survey design
The questionnaire was designed around each respondent’s evaluation of the most recent single market research study they commissioned. To ensure clarity of information and reduce burden, respondents were restricted to reporting on either a qualitative or quantitative study, but not a study with mixed approaches. Brief descriptive data were captured on the project (and the vendor providing the market research solution).
A set of 88 attributes was developed through internal brainstorming sessions with consultants at Observant LLC and with feedback from key clients. To simplify the questionnaire, the attributes, which may or may not describe the recent study, were presented to respondents by the phase of the project and grouped as proposal/design; research materials; respondent recruitment; field and data collection; moderation; analysis and interpretation; reporting; and vendor personnel. In some sections, alternative wordings of specific attributes were used to account for the same dimension across qualitative and quantitative studies. Additional interview moderator items and qualitative fieldwork were included for qualitative studies.
The project was then evaluated by completing a battery of attribute evaluations using the form: “How well does each of the following characteristics describe the [PHASE] of this research project?” Responses were indicated on an 11-point scale with three anchors: -5 (Inferior), -4, -3, -2, -1, 0 (Adequate), +1, +2, +3, +4, +5 (Superior).
Results
In all, 25 qualitative and 17 quantitative research projects were evaluated. Qualitative studies were predominantly marketing communications (44 percent), respondent motivation (24 percent) and product development (24 percent) studies, leveraging in-person in-depth interviews (72 percent) and telephone interview (44 percent) methods and including specialist physician (88 percent) and primary care physician (56 percent) participants. Quantitative studies were predominantly tracking (35 percent) and pricing studies (24 percent), leveraging Internet/online surveys (76 percent) and telephone interview methods (24 percent) and including specialist physician (71 percent) and primary care physician participants (47 percent). Some studies utilized more than one methodology or respondent category.
Top satisfaction drivers
Respondents indicated their overall level of satisfaction with the project on an 11-point 0-10 scale ranging from “Not at all Satisfied” to “Extremely Satisfied.” Qualitative projects tended to produce higher levels of satisfaction than quantitative projects (60 percent and 47 percent giving a top-2 box rating, respectively).
To identify the attributes most important in predicting overall satisfaction, bivariate analyses were followed by a stepwise regression model (n=42). This overall model did not consider items specific only to qualitative studies (interview moderation and fieldwork). The clarity of the report had the strongest association with satisfaction (R2 = .70). After this was first entered in the stepwise model, five additional variables were entered and no further explanatory variables could be identified (e.g., general vendor qualities such as overall friendliness and professionalism). Figure 1 illustrates the incremental contribution of each attribute to explaining the variance in overall satisfaction.
The most impactful attributes suggest that the key drivers of satisfaction are the level of empathy to the client and appreciation of their needs, coupled with intellectual sophistication; grasp of the business objectives; and an accurate, clear and complete report that reflects the client’s needs. On the more technical end, recruiting the ”right respondents” well and having precise research materials contribute to client satisfaction.
A second model was then run for qualitative projects (n=24), which included all the prior attributes plus the interview moderator and qualitative fieldwork attributes. The attribute with the strongest association with satisfaction was provision of partial/topline data as needed (R2 = .63). Eight attributes accounted for 95 percent of the variance in satisfaction, and any additional attributes each accounted for less than 1 percent of the variance. These top eight are shown in Figure 2.
Again sensitivity to, and anticipation of, the client’s needs are important along with a good understanding of the project objectives. Other more granular moderator variables such as “rapport with me and my team members” and “moderator’s skill in probing” failed to meet statistical criteria for inclusion. Similarly, qualitative field management factors failed to account for any meaningful incremental explanation of satisfaction (e.g., “quality of facility accommodations” and “vendor management of facility personnel and processes”). Some factors may be less visible to clients and may not factor into overall satisfaction. The only qualitative fielding factor attribute that was important was the provision of partial/topline data as needed. This likely reflects the vendor’s commitment to understanding and meeting the client’s needs – given the increasing pressure on clients to provide speedy feedback to their internal stakeholders.
Innovation
A theme within the industry at present is the need for innovation. Vendors are striving to come up with new approaches to capture data, build panels of respondents and harness the power of online social networks. Our results indicate that innovative approaches are simply not required of all studies. However, when required, innovation provided by suppliers is highly correlated with ultimate satisfaction.
Respondents were asked to rate how innovative the research approach taken by the vendor was, and how much the project necessitated a novel approach be taken, on 10-point scales. Innovation was demanded more of quantitative projects than qualitative projects (29 percent giving a top-2 box rating and 0 percent, respectively). Overall, the level of innovation observed was only weakly associated with overall satisfaction (r = 0.37, p = .017) across all projects, though this correlation rose to 0.68 (p <0.001) for projects where a level of innovation was expressly required (greater than the median on the need for innovation scale). Innovation in design was the final variable to account for meaningful incremental variance in the overall stepwise model. Consistent with the low-stated need for innovation in the qualitative studies, innovative design failed to account for incremental variance in the qualitative research stepwise model.
Fully satisfy the client
The results show that the overall drivers of satisfaction directly reflect the one-on-one relationship between the vendor company and the client/researcher buyer. How might this play out in practice? To fully satisfy the client, it is necessary to have a degree of rapport that facilitates sharing of the goals and objectives, but more importantly evaluating how the client’s needs will be met by satisfying those particular objectives. The market research supplier must have the acumen to appreciate the objectives and then proactively manage the project accordingly. .
And don’t forget an easy-to-understand, boardroom-ready report. The most effective reports most often arise from a strong relationship with the client. Relationships that foster effective communication lead to reports that demonstrate intellectual sophistication, provide insight beyond the data and clearly derive from the vendor’s anticipation of the client’s end needs.
Being reactive or behind the eight ball in foreseeing what is ultimately required can hamper the project and the client relationship. Attributes such as making maximal use of the data and having a moderator who is flexible in approach may be important in isolation, but in the absence of anticipating needs and appreciating the business objectives, may not prove sufficient to guarantee satisfaction.