9 types of research bias and how to avoid them

Editor’s note: Rebecca Sarniak is a moderating services specialist iModerate, a Denver research firm.

Seasoned research experts know that bias can find its way into any research program – it’s naïve to think that any research could be 100 percent free from it. But when does bias become a problem? And how do we identify and control the sources of bias to deliver the highest-quality research possible?

The goal of reducing bias isn’t to make everyone the same but to make sure that questions are thoughtfully posed and delivered in a way that allows respondents to reveal their true feelings without distortions. The risk of bias exists in all components of qualitative research and can come from the questions, the respondents and the moderator. To reduce bias – and deliver better research – let’s explore its primary sources.  

When we focus on the human elements of the research process and look at the nine core types of bias – driven from the respondent, the researcher or both – we are able to minimize the potential impact that bias has on qualitative research.

Respondent bias

1. Acquiescence bias: Also known as “yea-saying” or the friendliness bias, acquiescence bias occurs when a respondent demonstrates a tendency to agree with and be positive about whatever the moderator presents. In other words, they think every idea is a good one and can see themselves liking, buying and acting upon every situation that is proposed. Some people have acquiescent personalities, while others acquiesce because they perceive the interviewer to be an expert. Acquiescence is the easy way out, as it takes less effort than carefully weighing each option. This path escalates if fatigue sets in – some people will agree just to complete the interview. To avoid it, researchers must replace questions that imply there is a right answer with those that focus on the respondent’s true point of view.

2. Social desirability bias¹: This bias involves respondents answering questions in a way that they think will lead to being accepted and liked. Regardless of the research format, some people will report inaccurately on sensitive or personal topics to present themselves in the best possible light. Researchers can minimize this bias by focusing on unconditional positive regard. This includes phrasing questions to show it’s okay to answer in a way that is not socially desirable. Indirect questioning – asking about what a third-party thinks, feels and how they will behave – can also be used for socially sensitive questions. This allows respondents to project their own feelings onto others and still provide honest, representative answers.

3. Habituation²: In cases of habituation bias, respondents provide the same answers to questions that are worded in similar ways. This is a biological response: being responsive and paying attention takes a lot of energy. To conserve energy, our brains habituate or go on autopilot. Respondents often show signs of fatigue, such as mentioning that the questions seem repetitive, or start giving similar responses across multiple questions. Moderators must keep the engagement conversational and continue to vary question wording to minimize habituation.

4. Sponsor bias³: When respondents know – or suspect – the sponsor of the research, their feelings and opinions about that sponsor may bias their answers. Respondents’ views on the sponsoring organization’s mission or core beliefs, for example, can influence how they answer all questions related to that brand. This is an especially important type of bias for moderators to navigate by maintaining a neutral stance, limiting moderator reinforcement to positive respondent feedback that can be construed as moderator affiliation to brand and reiterating, when possible, the moderator’s independent status.   

Researcher bias

5. Confirmation bias⁴: One of the longest-recognized and most pervasive forms of bias in research, confirmation bias occurs when a researcher forms a hypothesis or belief and uses respondents’ information to confirm that belief. This takes place in-the-moment as researchers’ judge and weight responses that confirm their hypotheses as relevant and reliable, while dismissing evidence that doesn’t support a hypothesis. Confirmation bias then extends into analysis, with researchers tending to remember points that support their hypothesis and points that disprove other hypotheses. Confirmation bias is deeply seated in the natural tendencies people use to understand and filter information, which often lead to focusing on one hypothesis at a time. To minimize confirmation bias, researchers must continually reevaluate impressions of respondents and challenge preexisting assumptions and hypotheses.

6. Culture bias⁵: Assumptions about motivations and influences that are based on our cultural lens (on the spectrum of ethnocentricity or cultural relativity) create the culture bias. Ethnocentrism is judging another culture solely by the values and standards of one's own culture. Cultural relativism is the principle that an individual's beliefs and activities should be understood by others in terms of that individual's own culture. To minimize culture bias, researchers must move toward cultural relativism by showing unconditional positive regard and being cognizant of their own cultural assumptions. Complete cultural relativism is never 100 percent achievable.

7. Question-order bias: One question can influence answers to subsequent questions, creating question-order bias. Respondents are primed by the words and ideas presented in questions that impact their thoughts, feelings and attitudes on subsequent questions. For example, if a respondent rates one product a 10 and is then asked to rate a competitive product, they will make a rating that is relative to the 10 they just provided. While question-order bias is sometimes unavoidable, asking general questions before specific, unaided before aided and positive before negative will minimize bias.

8. Leading questions and wording bias⁶: Elaborating on a respondent’s answer puts words in their mouth and, while leading questions and wording aren’t types of bias themselves, they lead to bias or are a result of bias. Researchers do this because they are trying to confirm a hypothesis, build rapport or overestimate their understanding of the respondent. To minimize this bias, ask questions that use the respondents’ language and inquire about the implications of a respondent’s thoughts and reactions. Avoid summarizing what the respondents said in your own words and do not take what they said further. Try not to assume relationships between a feeling and a behavior.

9. The halo effect⁷: Moderators and respondents have a tendency to see something or someone in a certain light because of a single, positive attribute. There are several cognitive reasons for the halo effect, so researchers must work to address it on many fronts. For example, a moderator can make assumptions about a respondent because of one positive answer they’ve provided. Moderators should reflect on their assumptions about each respondent: Why are you asking each question? What is the assumption behind it? Additionally, respondents may rate or respond to a stimulus positively overall due to one factor. Researchers should address all questions about one brand before asking for feedback on a second brand, as when respondents are required to switch back and forth rating two brands, they are likely to project their opinion on one attribute to their opinion of the brand as a whole.

Bias in qualitative research can be minimized if you know what to look for and how to manage it. By asking quality questions at the right time and remaining aware and focused on sources of bias, researchers can enable the truest respondent perspectives and ensure that the resulting research lives up to the highest qualitative standards.
 

¹Dodou, D., & de Winter, J. C. F. (2014). Social desirability is the same in offline, online and paper surveys: A meta-analysis. Computers in Human Behavior, 36, 487–495. doi: 10.1016/j.chb.2014.04.005. https://www.iser.essex.ac.uk/research/publications/working-papers/iser/2013-04.pdf 

²Habituation of event related potentials: a tool for assessment of cognition in headache patients Neelam Vaney, Abhinav Dixit, Tandra Ghosh, Ravi Gupta, M.S. Bhatia Departments of Physiology and Psychiatry, University College of Medical Sciences & G.T.B. Hospital, Dilshad Garden, Delhi-110095, http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2605404/.

³Essentials or Marketing Research, An Applied Orientation By Naresh Malhotra, John Hall, Mike Shaw, Peter Oppenheim. Pp 227. http://www.readexresearch.com/understanding-survey-data/.

⁴http://psy2.ucsd.edu/~mckenzie/nickersonConfirmationBias.pdf; http://www.anderson.ucla.edu/faculty/keith.chen/negot.%20papers/RabinSchrag_ConfirmBias99.pdf UCLA

⁵Pirkey, W. (2015, May 6). Personal Interview. 

⁶Essentials or Marketing Research, An Applied Orientation By Naresh Malhotra, John Hall, Mike Shaw, Peter Oppenheim. Pp 227.

⁷Halo effects in consumer theories, Master Thesis, Erasmus University Rotterdam, thesis.eur.nl/pub/11759/Luttin,%20L.V.%20(352879ll).pdf