Editor's note: Dave Ensing is vice president, VOC integration, at Maritz Research, St. Louis. He can be reached at dave.ensing@maritz.com. This article appeared in the September 12, 2011, edition of Quirk's e-newsletter.
From time to time customer experience managers will hear the following questions from their internal clients: Is our response rate too low? What can we do to increase our response rate? Should we provide an incentive for people to respond?
Like many things in research, these relatively simple questions have somewhat complex answers.
When faced with these questions, the first thing to address is what issue is really being raised. Is the question really about increasing response rates (the percentage of people who respond to a survey invitation)? Is it about increasing the total number of responses at a given level of the organization (e.g., dealerships)? Is it about improving the representativeness of the responses obtained because of concerns about non-response bias? Improving the response rate is often not the most effective way to increase the total number of responses and/or improve representativeness.
The sampling scheme
To increase responses at the unit level and improve representativeness, the first place to look is the sampling scheme. Is the program sampling only a small percentage of customers in an attempt to control costs? If so, it is often more economically feasible to sample more customers and not use an incentive than it is to provide an incentive to increase response rates of a smaller sample.
Another aspect of the sampling scheme to examine is whether important segments of customers are being excluded from the sample frame. For instance, in the automotive industry it has typically been the practice that customer-pay (as opposed to warranty) customers are excluded from dealership service experience surveys, even though most dealerships do much more customer-pay service work than they do warranty work. This practice started because of difficulties getting access to customer-pay records. Now that systems are in place for most manufacturers to obtain customer-pay records, these customers should be included in the sampling frame. Obviously, inclusion of these customers will increase representativeness of the returns because an important part of the dealership's business will now be included in the responses.
As response rates decline the potential for non-response bias grows. However, it is not inevitable that studies with relatively low response rates have non-response bias. Rather than jumping to a solution designed to increase response rates (which, if done improperly, may actually increase non-response bias by encouraging some customers to respond more than others), clients are advised to conduct non-response studies to determine if non-response bias is a problem.
Cost-benefit decision
If the question is indeed about improving the response rate or if improving the response rate is likely the best way to improve the representativeness and/or the number of responses, providing an incentive to customers to respond is only one tactic. But again it is often not the most effective.
The choice of whether to respond to a survey invitation is a cost-benefit decision for the customer. How much will completing the survey cost the customer versus the benefit s/he will receive. At first glance, one might think that there is no cost to the customer to respond. However, there are many costs and these costs have been increasing over the past few decades.
Time. People are now more pressed for time than in years past and they are solicited more often for research than previously.
Effort. Many surveys are long and complicated.
Hassle/Boredom. Some customers feel duped by agreeing to take what they think is a short survey only to find out it is quite long; many surveys contain boring and repetitive questions.
Privacy. Many customers worry their information will not be kept confidential.
Spamming. Many customers are concerned their contact information will be sold to other companies and used for marketing purposes.
Subjected to a sales pitch. With the increase in selling under the guise of research ("sugging"), customers are more skeptical about the legitimacy of survey invitations.
On the benefits side of the equation, in years past customers often felt special and valued because they were being asked for their opinions. Unfortunately, as survey research has proliferated, being asked for your opinion is no longer a unique experience that conveys "specialness." Customers also seemed more motivated to contribute to the greater good by providing feedback about products and services than they are today. Some argue that the younger generations are less interested in the greater good and have even labeled Generation Y the what's-in-it-for-me generation. Also, those interested in providing feedback now have many ways of doing so (e.g., blogging, posting comments at customer-generated media sites, etc.) instead of completing a survey.
To increase response rates, researchers should look at both sides of the customer cost-benefit equation by seeking to decrease the cost to the customer and increase the benefits of participation.
Coordinate customer touchpoints. Many companies inadvertently over-survey their customers because different departments or divisions conduct independent research programs.
Make the task as easy as possible. Keep the survey clean and uncluttered with sufficient white space.
Make the survey as short as possible, but not shorter than necessary. Sometimes customers can interpret a very short survey as the company not really being interested in their opinions and just going through the motions of gathering customer feedback.
Keep it interesting. Make the survey as interactive and entertaining as possible, while maintaining collection of valid information.
Provide options. Give customers the opportunity to choose how and when to respond.
Include open-ends. Give customers the ability to tell their story rather than only answering a large number of specific closed-ended questions. Then use text analytics to gather insights from the customers' comments.
Be honest and thorough. Be very specific about how the information will and will not be used.
Include only the necessary. Avoid nice-to-know questions that are often included because customers are already responding anyway.
Avoid sensitive questions unless absolutely necessary. If they must be asked, explain to the customer why you are asking the questions and what will be done with the information.
There are also several ways to increase the benefits of participation to the customer.
-
Send customers a thank-you and briefly explain how the information is used.
- Show customers how the information they provide is being used to make improvements. For example, some companies have posted signs in their retail outlets telling customers what they are working to improve due to customer feedback.
- Assure customers they will get a personal follow-up if they request it and they will not get a follow-up if they don't request it. It is very important companies follow up on these promises. Otherwise, it will cause dissatisfied customers to become even more upset.
- Consider allowing customers to see other customers' feedback. People are social beings and they often want to know if their experience was typical or atypical.
- Provide an appropriate reward with monetary value to respond.
A monetary incentive
In most circumstances, to increase response rates we recommend investigating non-monetary methods before considering use of a monetary incentive (or any incentive with monetary value, such as a free oil change or a discount coupon for your next purchase). If not done properly, monetary incentives have the potential to bias the responses. This brings us to the issue of what makes an incentive appropriate.
If possible, provide the incentive to everyone being sampled rather than promising an incentive to those who complete the survey. In the case of cash incentives to complete a mail survey, most research has shown that inclusion of a small amount (e.g., $1) is more effective at increasing response rates than promising a larger amount (e.g., $5) upon return of the survey. There are many potential reasons for this but probably the largest is customers' skepticism that they will receive the promised reward.
The incentive should be something of equal value to everyone, regardless of their experience. Incentives such as discount coupons for the next purchase or the promise of a free oil change have two major problems. First, they are more valuable to people who intend to return to the retailer (e.g., those that previously had a good experience) than those who are unlikely to return. Therefore, they can bias the results. Second, they can be seen by customers as another marketing ploy.
Finally, the incentive must match the methodology and the geography. Inclusion of a dollar bill with mail surveys is relatively easy in the U.S. but it is obviously difficult to do for online or phone surveys. It is also difficult to include money in Canadian mail surveys because the one- and two-dollar currencies are coins and the added weight of including them increases postal rates.
A quick look
The following is a quick look at various monetary incentive options.
Including a dollar bill with a mail survey. Surprisingly, when using monetary incentives, this is still one of the most effective ways to increase response rates for mail surveys. This technique is particularly appropriate for small survey programs but can become financially infeasible for large programs.
Entry into a lottery to win a large prize upon return of the survey. For mail surveys, this technique is generally not as effective at increasing response rates as including a dollar bill with the outgoing survey. However, for large programs it is often more economically feasible than using a one-dollar incentive. For smaller programs it is less economically feasible. The use of a lottery is also easier to implement with online and telephone surveys. There are numerous laws and regulations concerning the use of lotteries as an incentive and it is strongly recommended that a professional promotions management company be employed to manage the lottery.
Including loaded cards. Loaded cards, whether they are phone cards, restaurant cards, store cards, etc., often provide a good way to incentivize response. One advantage these cards have over including cash as an incentive is that there is always some level of breakage - some customers do not use the cards. However, researchers must be careful that whatever card they use is appealing to customers being surveyed. For example, a card for $5 at Starbucks probably would not be appealing to customers who are not coffee drinkers.
Providing discount coupons. As discussed above, this is generally discouraged because of the potential to bias results and be seen as a marketing effort.
Contributing to charity in the customer's name. In general, this technique is not as effective at increasing response rates as either the dollar bill or lottery alternatives. If considering this alternative, it is important to include a number of relatively different charities the customer can choose from. Otherwise, the potential to bias the sample will increase because those in favor of the charity's cause might respond at higher rates.
Not to be made lightly
Including a monetary incentive for customers to return experience surveys is a decision that is not to be made lightly. It is fraught with potential problems. In general, non-monetary ways of improving representativeness, the number of surveys returned and response rates should be explored before considering incentives with monetary value. When considering monetary incentives, it is important to match the incentive to the size, methodology and geography of the program. Conducting a pilot test assessing the costs (both financial costs and results bias) and benefit (in terms of increased response rates) of several different types of monetary incentives is recommended.