Q Report respondents opine on AI, 
pain points and future plans

‘Aware of everything, everywhere, all at once’

If there’s one thing researchers are sick of, it’s hearing about AI. At least that was the constant refrain from speakers, audiences and exhibitors across our four Quirk’s Events this year. Funny thing is, for as much as they are tired of it, they sure seem to be curious about it, based on some of the findings from our 2024 Q Report.

As part of our annual survey of Quirk’s readers – the findings from which we will report on in this issue and in more depth through regular features in each successive issue – in addition to their thoughts on the impacts of AI, for this iteration we also asked about pain points in being a researcher and conducting research, the biggest research-related changes they foresee for their organizations in the coming year and their general thoughts on the use of emerging tools.

The survey was fielded from June 17 to July 23. In total we received 1,504 usable qualified responses, of which 502 were from end-client researchers and used for this end-client report. An interval (margin of error) of 2.49 at the 95% confidence level was achieved for the entire study. (Not all respondents answered all questions.)

When it comes to AI (or “bloody AI” as one presumably U.K.-based respondent called it!) 44% said they have already integrated it into their research, with 16% planning to in the next year and 31% in the “still reviewing” stage. 

Related to that, when asked which emerging tools from a supplied list they planned to integrate in the next two years, 80% selected AI and machine learning, 48% cited real-time data analytics, 44% indicated integration of behavioral data and 20% synthetic data.

To get a sense of the level of dread (or lack thereof) toward AI, we asked for their takes on its potential impact on marketing research. Two-thirds said it would probably or definitely elevate the role of MR (49% probably, 17% definitely); 22% picked “probably puts the role of marketing research at risk;” and 10% said it won’t have much of an impact.

Somewhat in contrast to that fairly sunny outlook on AI, comments to a related open-end were gloomier, ranging from the fatalistic…

I don’t think that AI (which is just a marketing term) can ever replace what we as analysts actually do (or are capable of doing if given the opportunity). But the hype, hope and push towards the cheapest, lowest common denominator may do real harm to the industry. The biggest benefit of [marketing research] is the custom aspect, getting insights specific to the business issue at hand and addressing the decision point directly. All this push toward commoditization just leads toward crappy outcomes for everyone (fewer MR jobs, poorer data and less informed decision-making). But the tech guys will pocket a bunch of cash before it comes crashing down.

I see significant risks to traditional MR when it comes to AI. I am hearing more and more of “Actually, Chat GPT drafted a great survey,” or “That AI xyz had a great answer for creating good-enough personas, segments, etc.,” or “I can just paste this to Copilot and analyze the data – why do we even need so much time for analysis?” Synthetic respondents are generating quite the curiosity. Meanwhile, panel and data quality is continuously deteriorating and it’s becoming increasingly hard to distinguish between real people and AI bots. Not to be all doom and gloom but MR must reinvent itself as an industry (like many others) to survive.

I’d say it puts the role at risk in the short-term because I’m pessimistic that the focus won’t just be on automation and simplifying processes. I hear more about replacement through AI than I do elevation of processes. I generally think, based on what I’ve been exposed to, the direction of AI tool development does a lot to undercut the professionals who have dedicated their careers to enhancing consumer understanding. Clearly, this is a sore spot for me at the moment.

…to the optimistic (but clear-eyed):

AI (if it can deliver on the promises made) will likely have a big impact on the function. Inside client companies, DIY survey platforms were the first wave that democratized the insights tradecraft. That made the process of conducting quant research doable for the average marketer. They still needed guidance. They needed to understand best practices. They needed to understand the philosophy and underpinnings of good research. AI is the second wave and MAY provide the guidance and thought leadership to execute good research. If true, the average insights person is going to have to redefine the value they are bringing to the table. This disruption may bring good.

It depends on what your MR team is doing. If you’re spitting out rote tracking reports every quarter/month/etc., then AI absolutely puts your team/role at risk. But if your team is trying to synthesize and elevate a blend of primary, secondary, cultural trends, behavioral data, etc., into actionable insights for specific business needs/decisions, I think AI elevates that type of team/role. Insights and MR roles are likely going to have to evolve, but on balance, I think AI will elevate the industry. None of us got into MR/insights because we loved doing those big tracking reports! I think AI will free up many insights professionals to do more of what many of us love about being in insights – telling deep, human stories with data that will impact our respective businesses.

And some commenters weren’t afraid to get out their crystal balls to envision what an AI-influenced future would look like for researchers:

I can imagine a future beyond even synthetic data when multi-modal AI models have evolved to the point where we will engage with them exactly as we would any other intelligent agent – another human, for example. We currently rely on data (the rows and columns kind) to build knowledge and understanding of the world around us. In this future, our AI colleague is able to help us identify product, brand and marketing opportunities and to exploit those opportunities without our kind of data being part of the process. The AI of this future has gained knowledge and understanding from the world (real and digital) directly. In other words, our AI friend is aware of everything, everywhere, all at once.

While I don’t think anything can replace the human element, companies will look at the money and realize they don’t need people to do [research] any longer. It may come full circle; they may eliminate research positions but then when they see AI can’t replace them, they will bring them back. But the damage will be done at that point.

Others reflected on how the capabilities and presence of AI might affect how non-researchers within a company or organization view the insights-gathering function as a whole:

Everyone already thinks they can do research. This will just make [that feeling] more universal.

Similar to “democratization” of research, [AI] may shift some research responsibilities to non-specialists (potentially at the cost of data quality and methodological rigor).

There is a concern that others in the organization may feel they can simply use AI to get a quick-and-dirty answer on something that should have insights providing input and direction.

Anything that promises to be quicker and cheaper puts the current tools at risk.

Some view the rise of AI as a response to marketing research’s failings or failures (the whole “nature abhors a vacuum” thing).

There is an argument to be made that as an industry we have done so poorly with incentivizing and compensating humans to participate that we are forced to turn to AI.

Even today, a basic ChatGPT query about specific audience care-abouts, pain points, etc., can often generate information and insights that are not much worse than what comes out of weeks of qual research. To some degree, it has me worried about the quality of the research and insights (black box, AI hallucination) but then again, our panel-based research models are broken also due to fraud and quality issues. I am a lifelong researcher but quite frankly, I look forward to AI providing us with better tools than the traditional research methods that are slow, expensive and fraught with challenges.

Broadening out from tools and methods, we also asked respondents an open-end about the biggest research-related changes they foresaw their organizations making in the coming year. Along with several mentions of incorporating AI there were the usual expressions of the need to do more with less, either because of budget cuts or staff reductions, and the attendant determination to make better use of the data they already have.

Developing a resource library for stakeholders to access key themes. This is to help share our research with a wider base and to reduce repeat research projects for different parts of the business, when we may already have the answers they need.

Some of the traditional MR is getting lots of scrutiny and pressure because what people say they will do is not the same as what they actually will do. So marrying self-reported with marketing analytics and other passive data (like Adobe Analytics) would be an evolution we’d be pursuing more actively.

Also there were about the same number of expressions of plans to bring more projects in-house (therefore cutting back on the use of vendors) and expanding the outsourcing of work to research agencies.

We are deeply evaluating all of our partners for key research programs (i.e., tracking studies, agile tools, data collection/reporting platforms, etc.) and subscriptions to make sure we’re getting the most from our budget and working with partners who can be extensions of our teams.

We are moving away from doing the vast majority of work in-house and instead will be continuing to expand the number of projects we farm out to vendors and the number of research vendors we work with.

Many struck a more hopeful/excited tone when expressing their plans.

Expanding our knowledge and insights across the entire organization to have great impact. Implementing a consumer-centric culture and continue to lead our industry in all aspects of growth and brand differentiation.

More accessible insights for upper management. Research and insights haven’t been engrained in the business historically so we have been working towards standing up foundational studies to have more insights flowing through the business.

Use findings more strategically (hopefully), with stakeholders held accountable for acting on findings/recommendations.

We have a newer CEO that has been in his role for about a year and a half and there have been a number of restructures that have had impacts across the organization. We have implemented a new strategy process and as a result more research is being conducted and utilized than ever before. I see us having a lot more research requests come from the organization over the next year.

Paired with the forward-looking questions in the survey were some sections designed to get a current read of researchers’ situations, this time in the form of pain points they face in doing their day-to-day jobs. 

The responses to our list of possible pain points generally follow the age-old researcher lament of having more work than they can handle and lower budgets than they’d like. A combined 46% said “too many projects for our budget” was always or often a pain point. Related, a combined 57% cited “too many projects for our staff” as always or often a pain point. Other top pain points include “no consistent or effective way to measure value of completed projects,” “decisions that go back and forth and/or get made late or ineffectively” and the pressure to cut costs without reducing the quality of research.

Happily, finding and keeping good researchers was not a common worry, as a combined 57% said doing so was rarely or not a pain point. Ditto for unclear project goals and objectives (a combined 42% rarely or not) and staying up-to-date on research methods (a combined 41% rarely or not). We asked for elaboration in the open-end and beyond those that added color to the existing answer choices the responses roughly fell into three categories: vendors, internal factors and leadership.

Vendors

Vendor quality has gone down in a major way. My hunch is that vendors are reducing team sizes and stretching teams across more projects and accounts. As a result, my team has to very closely manage vendors who would otherwise field flawed surveys, use too junior of moderators and deliver incomplete or surface-level reporting (which often requires hours of polishing before sharing with leadership).

I’m sick of all the self-service platforms and dashboards and tools. I want quality respondents.

We are a corporate in-house research group that does most of the hands-on research. Hesitate to say DIY because that implies low quality. One key challenge is vendors’ dismissal of our abilities to be good researchers or be objective – which really comes across as competitive jealousy. The other is sometimes the difficulty of working with partners for less than full-service services.

Finding vendors that are willing to work with you and listen. So many believe they are the expert in methodology and do not LISTEN to their very experienced CLIENTS. We want proprietary research, not syndicated formulas. Another huge pain point is SAMPLE. Finding and validating qualified, REAL respondents, especially in B2B, is becoming extremely difficult. First lesson I learned when I started my career is that any error within a project can be corrected, EXCEPT BAD SAMPLE. I would like a partner to be a true partner and not caveat everything they say.

The inability to get past traditional methods and advance. It feels like our suppliers are just ready to die with lack of survey response vs. trying to find new ways to collect valuable data. This forces us to look at companies that offer some new untested gizmo but that don’t have traditional MR capabilities. Would be great to find a company that did both the tried and true (but dying) aspects of traditional MR while innovating with new capabilities.

I think one of the pain points that I have encountered over the last couple of years is research vendors who are not paying respondent incentives in a timely manner. I see and hear way too many that are taking weeks to get incentives into their research company accounts and then several more weeks to get digital gift cards to the respondents. This is completely unacceptable as a corporate researcher. If digital gift cards are used, they should be able to be sent same-day or at the very latest 1-2 days after the session. If some companies can do it, then the other companies are just holding onto the advance payment of incentives for their own cashflow problems. Ultimately, I believe that this is going to impact the quality and availability of respondents for the entire industry.

Internal factors

One of my biggest pain points is just getting stakeholders in the same room. I have projects that can take 6-7 read-outs and attendance can be an issue. This means not everyone has the information despite facilitating over e-mail. Cross-functional teams not intaking the information together leads to issues in executing actions.

It’s never about the research but only about 5% of researchers get that. Until they do, they’ll be compartmentalized as a nice-to-have tool instead of a vital business function.

Many tasks are still highly manual and take time away from strategic thinking and drawing conclusions. We should be able to leverage technology more to speed up things like questionnaire development, data collection and validation and report-writing.

Researchers who are used to the old/traditional model of research, i.e., only managing vendor-driven research (with big budgets and timelines). They see pivoting to new agile/DIY models as “beneath them” – even though this is where the industry is going.

Our company just implemented an AI task force that must approve every vendor that uses AI in any way, even if they don’t use our proprietary data and this is slowing down our current research process.

Very limited budget. I make recommendations on when and how to do survey work and it is sometimes ignored, producing low response rates that they then blame me for.

Leadership

We have siloed business units/teams that often duplicate efforts – we have many different research/insights functions across different parts of the company, so making sure that we’re all talking to each other and not doing the same thing can be a challenge.

Extremely bad communication of expectations (of job role, swim lanes, project needs, etc.) due to poor leadership.

Senior management dismissing the need for research and spreading that thought to his direct reports.

I’m in an org that likes to hire consultants who fake research expertise then I am called into save it. I was asked two days ago by one of them, “What’s an IDI?” How about if they just say they are going to have meetings, not focus groups? Don’t call that crap research.

And finally, here were some pain points that maybe aren’t pain points at all!

Not enough time to meet all stakeholders that are waiting to learn about consumer insights and learn from us.

For me, as a client-side researcher, the biggest pain point is that the research is so valued that it is also feared.

Our organization is a little unique from what I’ve experienced and seen in the past 25 years, in that we are making MORE of an impact across the org, not less, and senior leadership recognize and prioritize our work as a differentiator in the market. Somehow, I found work with a company that is a true unicorn in this way.

Here’s to finding more unicorns!