Making sense of it all

Editor's note: Jim Longo is co-founder and chief strategy officer at Discuss. He can be reached at jim@discuss.io.

Artificial intelligence (AI) is taking the world by storm as one of the most exciting, and sometimes controversial, technology trends. The latest tool getting all the tech buzz is OpenAI’s ChatGPT which employs chatbots to write essays, poems, e-mails, etc., and is one of the fastest growing apps, surpassing one million users within a week of its launch. According to Grandview Research, the global AI market is currently worth $136.6 billion and is expected to grow to $1.81 trillion by 2030. 

While AI has been around for decades, it has never really lived up to the expectations until more recently. Why is that? The technology industry has evolved thanks to increased data volumes, advanced algorithms and improvements in computing power and storage, making it easier and faster for computers to think and act like humans.

As businesses become more inundated with data from various sources, gleaning insights from this data can be an overwhelming task. AI is helping to solve one of the biggest problems in the market research industry: culling through thousands of interviews and transforming disorganized raw data into powerful intel. 

Researchers are beginning to experiment with AI to help turn unstructured data into actionable insights. The desire to draw meaningful insights from business data is not new but the large volume of data being generated every day has made traditional processing methods ineffective. In addition, much of this data is more structured like spreadsheets with rows and columns of data that is easier to analyze. For example, a spreadsheet of point-of-sale data can easily reveal which stores are performing well and which ones are not.

Easier to interpret

In the world of quantitative research, data is interpreted using mathematical algorithms and models. This kind of black-and-white, structured data is easier to interpret and unveil. Although these models may be useful in providing facts and statistical data, they are more limited in their ability to provide insights about subtle trends and nuances. 

  • Qualitative research, on the other hand, involves the collection and analysis of non-numerical, unstructured data such as words, images, videos and observations. It’s often used to gain a deeper understanding of people's experiences, opinions and behaviors. However, there are several reasons why it is difficult to scale qualitative research, including:
  • Qualitative research often relies on small, carefully selected samples. Researchers may conduct a focus group with a small number of people in order to get a deeper understanding of their experiences. While this approach can be effective, it makes it difficult to generalize the findings to a larger population.
  • Qualitative research can be resource-intensive. It is estimated that over 80% of a research team’s time is spent on logistics such as coordinating, editing and analyzing conversations when it should be spent on strategy and decision-making. Researchers may also need to conduct multiple interviews or focus groups or spend long periods of time observing people to gather the data they need. If a researcher conducts 50 video interviews and then needs to sift through each one to summarize the key findings, it could take up to 50 hours. This can make it difficult to gather a large amount of data in a short period of time. 

While numerical data from quantitative research can be easily quantified and compared, qualitative data is more subjective. Different researchers may see the same data in different ways, making it difficult to reach consensus on the meanings and implications of the data. It can also be harder to draw reliable conclusions from qualitative research, especially when the data is collected on a large scale.

At the same time, more non-researchers are getting involved in conducting research in the product and CX space and may see AI as a shortcut to getting faster results. To ensure the findings are truly actionable, it will still require the role of someone trained to question what was not said, as oftentimes this uncovers the true emotions or drivers of behavior. 

For too long, product, design, CX and marketing teams have not had the technology to support their desire for a way to scale qualitative feedback – whether moderated or self-captured – from customers. Instead, they have had to navigate a complicated and long-winding path that dramatically limits both the number of customer perspectives that can be collected and analyzed as well as the specialized expertise needed to perform this research.

Accelerated the need

While the technology for online research and panels has been around for years, the shutdown of travel and in-person meetings during the pandemic accelerated the industry’s need to conduct their customer experience research remotely. Many companies relied on piecing together online meeting platforms like Zoom and Teams to conduct their market research. Unfortunately, these cobbled-together solutions were not designed for research and often require more time and resources to implement. Further, they don’t have the features and functionality to provide a seamless experience.

However, qualitative research doesn’t need to rely on timely, costly and outdated processes. New innovations and approaches to qualitative market research are making it easier for companies to gather feedback and deliver more meaningful and actionable insights to their customers.

The adoption of software-as-a-service (SaaS) and automation is democratizing and accelerating customer research. Today, the availability of online qualitative tools has created a major shift away from traditional market research through automation and streamlining workflows, cutting down on resources and administrative burden. In addition, purpose-built platforms are putting the power of automation into the hands of the research team, enabling them to spend less time on administrative tasks and more time on getting value out of the data.

AI-powered sentiment analysis, text-based analytics and video feedback tools are providing scalable insights into customers’ experiences that are transforming relationships between organizations and their customers. When used effectively, AI can be a tool to streamline the analysis on what was said and can save marketers hundreds of hours.

Themes and patterns

Discovering and sharing customer insights is now easier, thanks to a subset of AI called natural language processing (NLP). NLP’s automatic translation function quickly analyzes vast amounts of language with accuracy and precision. This eliminates the need to manually transcribe focus group discussions and can also capture verbatims delivered in multiple languages, which can then be processed to find themes and patterns.

Another benefit to using NLP for qualitative research is the ability to process people’s feelings and sentiments around key concepts and search terms. This function, called sentiment analysis, scans textual data to determine whether that data is positive, negative, mixed or neutral. Researchers can then boil down people’s feedback to underlying emotions and associations with their organization’s brand or products.

Qualitative research – for instance, observations and interviews – relies on capturing trends and themes and then assigning them to specific categories. This comes in handy especially when dealing with limited unstructured data that may otherwise be ignored by the big data algorithms. For example, in a business setting, occurrences can be grouped into internal, controllable factors, such as compensation or conflict management, and external, limited control, such as employees leaving because of poor housing or schools. If a specific occurrence is exhibited by most of the employees, then qualitative research is best suited to illustrate this trend.

By capitalizing on the key benefits of big data of providing general insights about occurrences, the intrinsic benefits of qualitative research and providing specific trends about specific occurrences, businesses can truly be smarter.

These AI techniques enable organizations to reduce market research costs, decrease manual work by shrinking the timeline from asking questions to identifying learnings and connect key data sources from multiple touch points.

In addition, it can not only process qualitative data from current research projects but it can mine past and historical data as well. This means that, if a researcher has not found an answer to an inquiry within current projects, they can search previous projects to find the needle in the haystack.

Entire life cycle

While technology and AI are transforming qualitative research, companies still need tools to support the entire research life cycle. Turning unstructured data into actionable insights starts with good-quality data and the ability to quickly organize, conduct and analyze qualitative research. 

What will be the ChatGPT of the market research world? Only time will tell, but within the next couple of years, the old ways of gathering insights will be shifting and AI is going to play a big role. It could be a real game-changer in not only automating processes but also further summarizing and distilling the bigger takeaways that will impact business decisions. If your company is not adapting to the latest research tools and trends, you risk being left behind.