Editor’s note: Bill McDowell is COO of Accelerant Research, Charlotte, N.C. This is an edited version of a post that originally appeared under the title, “AI’s biggest impact on market research should terrify us all.”
The market research and insights industry is buzzing with excitement over newly emerging and horizon applications of artificial intelligence (AI), and rightfully so. Machine learning, when applied to open-end coding, data processing, survey programming/administration, participant sourcing and even quantitative/qualitative interviewing is sure to have a lasting impact on helping us deliver faster, higher quality insights at lower costs. However, arguably the biggest impact that AI has had on the insights industry to date is one that all research practitioners and end-users should be very afraid of, and one that we’re not talking nearly enough about: survey bots.In a nutshell, survey bots are programs/algorithms that allow for automated completion of online surveys. In most cases, bots are a means for unscrupulous programmers to rack up cash, rewards and sweepstakes entries for participating in paid research studies without actually taking the time to share honest opinions. On the surface, these participants look to have qualified for and legitimately completed a survey, but the validity of their survey data is complete garbage. Bots are becoming more prevalent and more sophisticated. Our industry should be obsessed with weeding them out. For a fun, fear-inducing exercise, Google “survey bot.” What you would hope to see in your results is a collection of market research and academic thought leadership on how to avoid and prevent such parasites. However, what you actually see is a laundry list of available tutorials, services and software downloads for executing your very own bots and gaming the system.
It’s certainly a scary issue but not an insurmountable one. Clients and suppliers need to remain vigilant and work together to minimize the impact of survey bots. Here are three relatively painless ideas for minimizing impact:
- reCAPTCHA – The “I’m not a robot” widgets that we see on so many e-commerce and social media sites are easy to insert into most online surveys. If you’re using a survey platform that doesn’t allow it, it may be time to start shopping for a new provider.
- Quality control questions – A great method to help control bots (as well as to keep human participants on-task and engaged) is to add short quality control questions to your survey. Open-ended questions and multimedia (e.g., asking respondents to listen to an audio snippet and respond) tend to be more difficult for bots to overcome.
- Rely on sample providers who are equally obsessive with data quality – Any reputable online sample provider should be working on your behalf to prevent bots from making their way into your survey. Additionally, any that do slip through the cracks should be removed and replaced at no additional charge.
Unfortunately, there is no catch-all solution to stopping survey bots, which is why it’s important to take quality assurance measures during all study phases, from sample procurement to survey development/programming to data analysis. As an industry, the best thing we can do is collectively admit that we have a problem and partner with one another to make sure we keep this issue at bay.