Editor's note: James A. Rohde is consultant and founder of James A. Rohde Consulting, a Pittsburgh research firm. He can be reached at 412-589-9572 or at james.rohde@jamesarohde.com. Michael Lieberman is founder and president of Multivariate Solutions, a New York research firm. He can be reached at 646-257-3794 or at michael@mvsolution.com. This article appeared in the January 9, 2012, edition of Quirk's e-newsletter.
When we ask people (employees, customers or any other kind of respondent) for their opinions, one of the key points in ensuring reliable answers is an objective question. Having an objective question includes the words within the question itself but also the environment in which it is asked and who is doing the asking. The whole point is to make sure that when a question is asked, there is not an answer that seems "better" than another.
This is important because people have the tendency to answer questions in way that allows for a more positive self-reflection. This tendency exists between people but, more importantly, it varies between the situations that people are in when asked the question. This is the reason researchers insist on how and where questions are asked.
This tendency, referred to as social desirability, does not mean respondents are necessarily going to be untruthful. It does mean that you may only be hearing more about their overly-positive self-reflection than an assessment of how your respondents actually behave.
This is not necessarily new information when you consider that most respondents will tell you brand name is not important and that each purchase decision is a careful calculation of value. There are many tools for getting around this issue but we thought it would be fun to share how this tendency is actually measured.
If you are interested in seeing where you fall, take the 20-question quiz below before reading the scoring key. This is a test that is used by psychologists when studying behavior and has been in use for a little over 20 years. For what it's worth, we would not necessarily recommend giving this quiz to customers as a way to assess their answers.
Social desirability quiz
Indicate the extent to which you agree with each of the following statements. Use a seven-point scale to indicate your response, with 1 = "Not True" and 7 = "Very True."
1) I sometimes tell lies if I have to.
2) I never cover up my mistakes.
3) There have been occasions when I have taken advantage of someone.
4) I never swear.
5) I sometimes try to get even rather than forgive and forget.
6) I always obey laws, even if I'm unlikely to get caught.
7) I have said something bad about a friend behind his or her back.
8) When I hear people talking privately, I avoid listening.
9) I have received too much change from a salesperson without telling him or her.
10) I always declare everything at customs.
11) When I was young, I sometimes stole things.
12) I have never dropped litter on the street.
13) I sometimes drive faster than the speed limit.
14) I never read sexy books or magazines.
15) I have done things that I don't tell other people about.
16) I never take things that don't belong to me.
17) I have taken sick-leave from work or school even though I wasn't really sick.
18) I have never damaged a library book or store merchandise without reporting it.
19) I have some pretty awful habits.
20) I don't gossip about other people's business.
Scoring key: Give yourself one point for each 1 or 2 response to odd-numbered items and one point for each 6 or 7 response to even-numbered items.
The test developers found a mean score of 4.9 and a standard deviation of 3.2 for female college students and a mean score of 4.3 and a standard deviation of 3.1 for male college students. Those who score higher are more likely to present themselves in a more positive light.
Typically unaware
It is important to understand that people who are more prone to present themselves more favorably than others are typically unaware that they are doing so. If we look at the different items on this test, few of us can say that we have absolutely never tried to cover up a mistake. However, somebody who rates high on social desirability may be more inclined to exaggerate the truth and indicate that this point is true for him or her.
Now, if we did actually know which of our respondents rated high for this tendency we may be able to adjust our interpretation of the results. But honestly it would likely be a challenge to get this information in most, if not all, of our studies. What is more useful though is to understand that this tendency exists and that it is a matter of to what extent respondents are prone to it, not if they are.
So when we ask a question such as, "To what extent do you make a purchase on impulse?", a low score could reflect a desire for the respondent to present themselves in a rational light just as much as it could reflect behavior. As a result it is more difficult to interpret the results to this question than it would be for a question such as "How often do you make purchases that are only on your shopping list?"
Remains on the front burner
In market research, perceptual validity of questions has been the subject of many discussions. There are certain methodologies to correct for social desirability (e.g., division by the standard deviation). However, the issue of social desirability, and data validity as a whole, remains on the front burner of branding image and product research and likely will be as long as research relies on Q&A.
References
Lockard, J.S. and D.L. Paulhus (Ed.). Self-Deception: An Adaptive Mechanism. Prentice-Hall, 1988.