Exploring AI-powered conversational surveys with ŌURA
Editor’s note: Dale Evernden is head of UX and founding partner at Rival Technologies.
With technology advancements and client needs evolving at breakneck speeds, market researchers need effective innovation programs.
As they become increasingly reliant on technology-driven methodologies and agile approaches, researchers must create formal, structured programs that are ideally built in collaboration with customers, partners and other stakeholders. In this way, bespoke experiments can lead to the co-creation of new capabilities directly aligned with specific needs and challenges.
And while innovation can be daunting, it’s our responsibility as insights professionals to ideate, iterate and collaborate. As part of this effort, we must not shy away from our mission to experiment with AI. Let’s look at how we can maximize our chances of success.
Breaking ground with AI: The rules of engagement
As AI becomes an increasingly common part of our daily lives, we often watch with mixed emotions – fear, uncertainty, doubt and awe. In fact, research from our sister company, Reach3 Insights, found that fears around AI have quadrupled in the past year.
This fear is seen in the insights space as well, with many researchers wondering, “Am I being replaced by an algorithm?” But the fact is, AI will not replace humans, although humans with AI will replace humans without it.
AI has arrived and it’s here to stay. We simply have to figure out how to work with it and how to get the best out of it. We know, for instance, that AI can enable:
- Automated data processing: AI can handle large data sets by cleaning, organizing and preparing data for analysis faster than traditional methods.
- Segmentation and personalization: AI can identify patterns in demographic, psychographic and behavioral data to create hyper-targeted campaigns.
- Text and sentiment analysis: NLP can process and analyze qualitative data from customer feedback or social media posts to gauge sentiment, which helps with thematic analysis.
- Data visualization: AI tools can generate visualizations that bring to life even the most complex data sets.
Case study: Testing AI probing research innovations with ŌURA
We co-innovated with ŌURA, maker of the industry-leading smart ring that translates the body's most meaningful messages (sleep, activity, stress and heart health), to transform how its 2.5 million users feel every day. With 10 years of research and development behind its design and technology, ŌURA continually collects data on over 20 biometrics that directly impact its members’ well-being.
Together, we conducted iterative exploration through AI-powered conversational surveys. Our hypothesis: Can we use AI to dynamically evaluate a single open-ended response and then probe with conversationally aware follow-up questions to collect a more thoughtful response?
We sought to answer questions such as:
- Is the tech feasible, in terms of speed, performance and reliability?
- Can the AI generate appropriate questions?
- Can we create an acceptable UX?
- Will probing actually deliver deeper insights?
During the experiment, we used an AI-powered tool that follows up on qualitative open-ended questions and prompts participants to share more. ŌURA saw significant improvements in feedback quality and relevance. It resulted in a 293% improvement in thoughtfulness score. Thoughtfulness score is a way to evaluate open-ended responses based on criteria such as depth of insight, relevance, specificity, clarity, coherence, originality, critical thinking, emotional and empathetic engagement, breadth of consideration, use of supporting evidence and constructiveness with AI.
The tool, which we’ve now dubbed “AI probing,” also got the thumbs up from research participants. In fact, 94% of participants said that the AI-generated questions they received were relevant and appropriate and 99% indicated that the AI-generated questions were easy to understand.
Best practices for today’s innovation programs
These types of iterative, collaborative exercises can be game-changing when it comes to delivering deeper, richer insights but there are some guardrails to follow:
- Start with a clear goal and test your approach. First, it’s imperative to test – ideally before reaching out to a real community. What’s more, always base your approach around an overarching, central idea or ultimate goal and keep that in mind. And don’t forget that, while our use of AI may never be perfect, we can still aim to move from, say, a four out of 10 to a nine out of 10.
- Choose secure AI tools tailored for business needs. ChatGPT arguably stole the limelight when it comes to mainstream adoption of AI, but, in the research sector in particular, it’s critical to use a secure OpenAI account. ChatGPT is unlikely to be appropriate for business-related experimentation or exploration. Fortunately, there are different subscriptions available, such as ChatGPT Plus or Team-GPT, which do not automatically train models with threads or conversations you have with it. The latter also enables the sharing of prototypes internally. We have found that sharing concepts and storytelling is critical.
- Ensure relevance with customization. The use of a simple prompt template can provide a great starting point for customization. For instance, “I’m responsible for delivering market insights on ‘x.’ As part of this work, I find it painful or awkward or difficult to ‘y.’ How might I use AI to improve x?” – and so on. In addition, OpenAI has also made it easy to prototype custom GPTs. But be sure to upload relatively clean data files – never forget the old adage: garbage in, garbage out.
- Don’t underestimate the role of humans. It’s important to keep a human in the loop. We must seek augmentation over automation and build with the intent of a human using the tool and AI adding value. After all, AI can develop some bad habits, such as algorithm bias, privacy and security breaches, ethical oversteps and more.
- Ensure transparency. We must always seek to maintain trust. If you use AI, reference it. Once trust is lost, it’s hard to get back. Finally, prioritize data privacy and compliance and consider indirect stakeholders. Ask yourself who might need to be aware that you’re using AI.
Building a culture of continuous improvement
As technological advancements such as AI-driven survey enhancements and qualitative data analysis tools transform the industry, a formal innovation program can enable rapid-fire progress. By testing, pivoting and testing again, insights professionals can build a culture of continuous improvement through experimentation and the refinement of ideas.
What’s more, by working closely with clients and industry leaders with a truly collaborative approach researchers can ensure that innovations are not only cutting edge but also impactful.
It’s only by embedding a systematic approach to innovation that we can hope to stay ahead in an ever-evolving landscape – and provide the insights brands depend on.