Is that your final answer?

Editor’s note: David D. Fish is strategic research manager for Toyota Motor Sales USA, Inc., Torrance, Calif.

Over the years there has been a fair amount of debate about how to increase response rates in consumer mail studies. This is only natural, since mail studies are increasingly expensive and boosting response rates just a few percentage points can mean saving thousands of dollars. This issue has become especially cogent recently, with the vast number of mail surveys competing for consumers’ attention.

The short test below is based on experience and academic research on the topic. Below is a list of commonly held beliefs about response rate improvement; some are fact, some fiction. While there are no absolutes in research, the preponderance of evidence seems to support the answers below. Test your skill in this short true/false test.

1) True or false: The use of color will increase response rates in mail studies.

False. Slapping some color onto an existing questionnaire will not increase response rate. Even a careful redesign that adds graphics and multiple colors tends to have a minimal impact. In fact, it may decrease your response rate. If your questionnaire is too glitzy it may be perceived as a direct mail piece and simply be tossed into the trash. However, that is not to say that your questionnaire should be sloppy or poorly designed. Questionnaires should look professional and official, with clearly-worded questions that are easy to complete. The layout of the questionnaire should also be simple to follow, without a maze of skip patterns and a menagerie of arrows pointing to separate parts of the questionnaire. The best advice: keep it simple and clean.

2) True or false: Stamps on return envelopes will increase your response rate over pre-printed postage-paid envelopes.

False. Intuitively, you may think stamps on an envelope rather than a sterile postage-paid marking would increase your response rate. That has not been my experience nor has it been reported in the academic literature. Since postage-paid permits are typically cheaper and require less labor (no one has to lick all those stamps!) you might as well go with the pre-printed postal permit approach.

3) True or false: The length of the questionnaire significantly effects response rates.

True. This can have quite dramatic effects on response rates. However, the word ”length“ can be misleading. If you squeeze eight pages of content into four pages, you will get an equally dismal response rate. The optimal approach is to triage your most important questions for inclusion into your questionnaire, then front-load your questionnaire with the easiest questions first. Alternatively, design some kind of logical flow for the questionnaire. For example, in a buying-process questionnaire you would want the questionnaire sections to correspond with the buying process. The bottom line is, the shorter the survey, the better the response rate - so choose your questions wisely.

4) True or false: A recognizable name brand will increase response rates.

You bet. The more well-known the brand name (e.g., Toyota), the more likely it will help response rates. However, the need for sponsorship anonymity may preclude this option. For example, if you are trying to get an honest evaluation of your product versus other products it would not be a good idea to include your company’s name on the cover letter.

However, if it is a quality survey for a product purchased from your company, the survey provides the added benefit of generating customer goodwill. A follow-up satisfaction survey on product purchase conveys to customers that the company really cares about their satisfaction, which may increase post-purchase satisfaction and possibly repeat sales.

5) True or false: Incentives to increase response rate are cost-effective.

Not usually. While incentives certainly work to get your response rates up, they are sometimes not as effective as simply buying additional names. However, that approach also has its drawbacks in the form of increasing the menace of non-response bias in your sample. Incentives are good to use in general, but particularly for hard-to-reach populations (e.g., vegetarian truck drivers). However, sometimes for the cost involved, incentives just do not make financial sense.

6) True or false: Greenbacks work better than non-monetary incentives.

Everyone loves money; survey respondents are no exception. In general, cash is the best way to get people to fill out surveys. While other methods (phone cards, lottery, etc.) also provide a bump in response rate, cash still seems to do the best job in increasing response rates. However, other approaches, such as giving away merchandise with your company logo, can provide the added marketing benefit of getting some exposure for your company. Again, you might want to consider this in a customer satisfaction setting.

7) True or false: “Pre-notification” postcards increase response rates.

There is limited evidence that the pre-notification post card increases response rates. When post cards have been shown to work, the effect is generally minimal. This modest response rate bump may not worth the money.

8) True or false: “Reminder” postcards increase response rates.

Reminder cards appear to have a slight impact on increasing response rates. However, you may want to weigh the cost of sending them out. If it is a hard-to-reach sample, or you are especially concerned about non-response bias, I would recommend this approach as good way to create a modest increase in your response rate for a nominal cost.

Serious business

Well, how did you score? Raising response rates is serious business and trial-and-error of techniques can be expensive. Using the points above as guidance will allow you to use your market research dollars more effectively.