Editor’s note: Graham Ruddick is co-founder of research firm Insightflow.io, London.
“The future is already here – it’s just not evenly distributed.” William Gibson, 2003
My team is trying to unravel some of the contradictions inherent in the quote above by conducting a series of interviews with various experts in the fields of AI and market research with a hope that we can, at least, help structure the conversation.
In many ways this discussion is part of a broader conversation about if and where AI will impact any business sector. However, market research has a number of characteristics that might make it a fertile marketplace for AI.
- Market research often has large or very large data sets available for analysis. Large data sets are very much a necessity for AI to thrive.
- Market research has a huge amount of human interpretation of data to understand evidence, create insights and write the business stories that are the product of good research. This human activity provides the training data sets essential for the effective use of AI.
- Market demands are creating a need for increasing research velocity. Business is asking for more evidence to be produced more quickly, driving the “...need to convert unstructured data to meaningful data and insight at speed and scale,” says Brian Livell, Big Sofa Technologies.
What can AI contribute?
Amid the many predictions about how and where AI might be brought into the MR world, there are a number of potential benefits that many of the community feel are plausible:
- Reduce the dog work for individual researchers. AI will allow researchers more time to do what humans are good at. “Companies that can combine AI-based technology with expert consultancy will have significant advantages in the industry over those that fail to embrace AI as a driving force,” says Livell.
- See and present patterns in larger data sets. As data sets grow, AI (when well trained) can rapidly provide structures and patterns that humans may be unable to see. In turn this may make previously unseen correlations visible. Although it will remain a human task to understand what those correlations mean.
- Provide a link between quant research and qual research. Big Sofa Technologies describe this process as technology and people as a service. Our conversations has shown the theme: humans do human. “In the bigger picture pieces of data should be seen as incitements to excitement. The implication is that the excitement comes from what humans can do with great data and in turn it’s great tools that can produce the input pieces of data,” says Ben Smithwell.
What are the issues with AI in research?
The fact that many in the MR community regard AI with some skepticism indicates that there are problems. The trick may be to understand those issues and bring that recognition into deciding how – and when – AI might be used.
Many people have insufficient understanding of both the potential and the limitations of AI. “People don’t understand well enough what it could deliver to specify what it should deliver,” says Jason More, Chattering Monkey.
The result is a lack of credibility for AI driven solutions. This is exacerbated by the fact that many practitioners have already had a bad experience with the gap between AI’s potential output and the reality of how projects have been delivered. “Whenever I read that AI has been used [on data] I am a great deal more careful about how I use it,” says Adrian Rhodes, AR Consulting.
There is also the hyperbole that comes with new technology entering a marketplace. Inevitably some of the claims are overblown … smart companies selling mediocre software to uneducated buyers.
Evangelists and skeptics
There is a risk that AI and the study of big data turns researchers from people driven by nuance and lateral thinking, to being people driven by counting things. Almost everyone is aware of the growing links between AI and research. There are many evangelists who see it as, at the very least, a tool that will help researchers do more of the things humans are good at. On the other side there are plenty of skeptics who will point to the need for large training data sets, a continuing lack of accuracy/nuance in AI results and the cost of implementation. There are also ethical, privacy and bias issues with AI that deserve closer attention.
It is a conversation that won’t go away. It is a conversation in which many researchers should be involved.
Acknowledgements
Thank you to Brian Livell from Big Sofa, Jason More from Chattering Monkey, Tim Deeson from Green Shoots Labs, Adrian Rhodes from AR Consulting and Ben Smithwell from Smithwell.