Overcoming the Biggest Threats to Market Research: Bad Data and Bad Actors

By Jonathan Ephraim | March 13, 2023

Reading time: 3 minutes

Abstract

A new e-book authored by market research experts at IntelliSurvey explores how bad data is becoming more prolific as cheaters and other bad actors grow more sophisticated. This overview explores the modern slump in data quality and how researchers can combat these threats.

Research Topics:: Data Quality | Demographic Analysis | Fraud Detection | Marketing Research-General | Sampling | Survey Research
Industry/Market Focus:: Research Industry
Content Type: E-Newsletter Article | Sponsored Article

Share Print

The Modern Slump in Data Quality

Surveys are incredibly valuable for businesses, helping them drive growth through understanding.

Decision makers expect these insights to come from honest, authentic respondents who represent a target population. However, this assumption does not always hold. Finding the right survey respondents has become an immense undertaking.

In the modern survey data landscape, studies are contaminated with fake data from survey farms, bots and other types of malicious actors. Additionally, researchers don’t always have access to the right audiences. Worse still, some surveys are poorly designed, failing to engage respondents or are fraught with response bias.

Even when research organizations follow what they believe to be the right methodologies, bad data can find its way into survey results.

For over 20 years, IntelliSurvey has helped brands launch tens of thousands of surveys resulting in around 20 million survey completions. Our experience minimizing the prevalence of bad data in our clients’ research led us to create an in-depth e-book, Overcoming the Biggest Threats to Market Research: Bad Data & Bad Actors, detailing the evolution of survey data, survey panels and bad actors, and various approaches to data cleaning. At the end of the e-book, you’ll find actionable recommendations you can implement to minimize the effects of bad data on your research and decision-making. Over the next four weeks, we’ll break down four key areas to understanding and protecting against bad data, including:

The importance of data accuracy in business intelligence.
Fighting survey farms: The battle for accurate business intelligence.
Low incidence can be a big source of bad data.
How experts are improving survey data optimization.

Why has data quality from survey results gone downhill?

As mentioned above, survey contamination comes from several sources. As the demand for survey data has grown, so too have the incentives that come with a successful survey complete. This has attracted groves of new sample companies and, unfortunately, fraudsters. While some of the increase in supply is good, researchers now face new data quality challenges.

Bad actors

Inauthentic and dishonest respondents often lie about their qualifications to participate in a survey.

This can manifest in several ways:

Individuals looking for the quickest path to the reward without providing thoughtful, accurate answers.
Coordinated survey farms, which are fraudulent teams working on a larger scale.
Fraudulent bots, which are designed to finish surveys while appearing as natural human participants.

Access to the wrong audience

When researchers don’t reach their specific groups or demographics, insights won’t reflect the true nature of the target population.

There are a few ways that researchers can end up accessing the wrong audience:

Convenience sampling, like tapping a brand’s social media followers to participate in a survey.
Self-selection, which leads to survey respondents who have a strong bias in one direction or the other – as well as a higher probability of malicious actors getting into the mix.
Insufficient sample size, which can be heavily skewed by fraudulent actors making up a larger portion of the answers.
Using just one sample supplier, which reduces the survey footprint.

To prevent the situations described above, it’s critical for researchers to vet and choose sample suppliers who not only have access to the right audience, but also have demonstrable sample quality.

Failing to design against response biases

Survey data can be contaminated when researchers fail to design their survey questions in a manner that lessens the impact of response biases, like:

Non-response biases, where some groups of participants fail to complete surveys at higher rates than others.
Observation biases, where respondents skew answers to align with the conclusion they perceive the researchers want.
Social desirability biases, where answers reflect popular opinion more than the respondent’s true perspective.
Acquiescence biases, where respondents are more agreeable in answers than they actually are in the real world.

Minimizing bad data in survey results

Intentional or not, bad data always finds new ways to contaminate survey results. Fortunately, innovations in survey design and data cleaning can reduce the impact of bad data on insights.

To learn about the latest in modern data cleaning approaches and for tips you can implement today to improve your data quality, check out our e-book Overcoming the Biggest Threats to Market Research: Bad Data & Bad Actors.

Author

Jonathan Ephraim is president and co-founder of IntelliSurvey. Ephraim has over 20 years of experience as a researcher and thought leader. He holds an AB with honors from Harvard, has worked for leading consulting firms and co-founded IntelliSurvey in 2001.