Editor's note: Rill Hodari is the founding president of the Association of Minority Market Research Professionals, Chicago.
I recently conducted a member survey for a local non-profit organization and as with many member surveys, the client opted not to offer an incentive. Instead the communication, endorsement and encouragement from organizational leadership acted as the stimulus to get people to participate. The result was that the actual number of responses was good but the data proved to be fairly positively skewed, which made me consider whether the actual invitation language and leadership endorsement influenced both which members completed the survey and what answers they gave. My next thought was that maybe a monetary value reward would have enticed a broader range of respondents to take and complete the survey. Lastly, how might an invite absent of organizational influence that explicitly requested both positive and negative feedback improve the range of attitudes captured? And without the leadership endorsement, what else could effectively motivate participation?
Variety of incentive programs
I have found that the no-incentive approach is most common in member surveys, employee surveys, on-site event evaluations and/or post-purchase or post-usage satisfaction surveys. These studies focus on captive audiences attending an event or those more psychologically loyal by virtue of their employment relationship, voluntary membership or recent user interaction with a service or product provider.
Other methods include low-value incentive programs that typically include raffles or points programs that target broader audiences and generally have very low and ever-declining response rates. Thus, these incentives may entice the ends of the attitudinal spectrum, either happy customers or dissatisfied customers and so an additional tacit incentive is really to advocate or vent.
Lastly, there are higher-value incentive programs targeting harder-to-reach, high-value professional respondents such as doctors, lawyers, creative consumers, etc. This last incentive program is a very specific use case and even though it is controlled to be low relative to the professional respondent earning power, it still may contain bias in that there are large sections of this specific audience not moved to participate by the incentive offered. Also the value of the incentive may vary by personal financial behavior. It might appeal most to those either valuing frugality or who are more of a spendthrift and these behaviors may again align to the psychographic ends of the respondent spectrum.
Bias impact
The bias for each of these programs may be different. For example, in employee retention or climate surveys, the leadership endorsement and follow-up managerial encouragement is likely to garner more positive or at least neutral responses rather than authentic critique. That is not to say that these data are absent of negative responses but rather the measurement of the scope of issues might be lower than actual. For member satisfaction and event evaluations the same authoritative influencer effect may bias results. This is likely due to emotional apathy and the attitude that minor concerns aren’t worth the time of participation. Unless the member concern is large, people may genuinely feel that if you can’t say something nice, don’t say anything at all. They may be thinking it is kinder not to participate and give authentic feedback, leaving the organization blind to these opportunities for improvement which, in aggregate, could be actually significant.
In addition, this bias of kindness is relevant for new product research in that the whole point of survey research is to measure the scope of the adoption. If some aspects of negative feedback are absent or entirely left out this objective is never accurately met.
Monetary value rewards – whether small or large – do tend to influence feedback to be more positive. If the respondent had a neutral opinion, the reward is likely to sway responses at least slightly to the positive end of any rating scale.
Alternative solutions
There are creative ways to handle issues with incentives. One such solution may be to have two or three randomly rotating invitations with varying language.
- One invitation encourages feedback in the typical friendly tone.
- The second survey invitation might acknowledge that if the respondent had a negative experience to any degree this feedback is still very important to the organization.
- The third invite might include language indicating that even if the respondent had a non-memorable, neutral experience, it is still important to capture their perspective.
- Also a single-question opt-out option might be sent as follow-up to the initial solicitation that allows them to indicate if their experience was nothing special for a given list of service or product attributes. This opt-out removes the barrier of time participation vs. apathy and gives more presence to this largely missing respondent base. This helps facilitate more in-depth qualitative research to understand how to shift their perceptions.
By scripting and randomly using each of these invitations separately in communication with the target audience, the researcher gives a more balanced approach to respondents’ attitudes around survey participation. All three messages in a single invite would not work since most potential respondents would not read a very long invite.
Lastly, where monetary value rewards are used in conjunction with the randomly balanced invitation approach, the rewards can be more targeted and compelling. Currently, when a study sponsor wants to give respondents the most freedom in reward choice, they offer a gift card or something similar. When this is done researchers miss the opportunity to integrate Web analytics into their online research. One option is to offer a small discount on a product the respondent may have recently shown interest in. In addition, the ability for a respondent to gift their incentive to another person may be compelling. While these are better and more personalized ways of offering a gift card, the important factor is that you offer something relevant to the respondent.
With survey participation rates continuing to decline, researchers are looking at many different ways to make participation less burdensome, more fun and more accurate. Using innovative incentive programs may allow researchers to get a foot in the door with respondents.