On the Rise

Late last July, when we began fielding the survey for the 2020 Q Report, we were all in the thick of the pandemic. Almost a year later, launching the survey for this year’s Q Report, there are tantalizing signs that we are coming out of it – along with the delta variant’s daily reminders that COVID-19 is not going away quietly. While in 2020 we asked researchers many questions to get at how they were adapting to the pandemic’s many impacts, this year we pivoted to a more forward-looking approach.

In other words, how do we feel about where we are and where we might be going?

The Q Report work life and salary and compensation study of end-client/corporate researchers is based on data gathered from an invite-only online survey sent to pre-qualified marketing research subscribers of Quirk’s. The survey was fielded from June 10 to July 16, 2021. In total we received 1,951 usable qualified responses of which 816 were from end-client researchers and used for this end-client report. An interval (margin of error) of 2.16 at the 95% confidence level was achieved for the entire study. (Not all respondents answered all questions.)

Along with a host of questions about department staff levels, years on the job, skill sets and job satisfaction (see the accompanying content for a deeper dive on those findings), we asked open-ends about: the biggest research-related changes their organization planned for the coming year; how valued they feel marketing research is internally and in the business world as a whole; and how they feel about the future of insights as a profession.

Hopeful and optimistic

No major themes emerged from the question about the MR-related changes planned for the coming year and the tone was largely hopeful and optimistic despite the current societal backdrop. Lots of talk of automation (of reporting, analysis, etc.), of centralizing (from small tasks like getting a handle on processes such as documentation and global standards to reorganizing the insights function) and of coming to grips with the impact of company-wide overhauls.

Internalizing market research activities (survey programming, fieldwork and data processing) in order to become more agile and deliver market research results and insights much more rapidly. Providing a 360-degree view on market knowledge and insights: becoming data source- and survey methodology-agnostic!

Continuing to reinforce the goal of establishing the team as a center of excellence that the entire organization should rely on. The practice has previously been that some teams went off on their own to launch market research projects with external vendors, which in some cases led to redundant work. The centralized approach will streamline market research projects that could be utilized across the organization, while ensuring that internal-trained market research professionals manage the process from start to finish.

Rather than being focused on market research, our team is growing into a strategist role. It’s likely more of our research will be outsourced and our team will be more focused on the ‘story of the data’ and how it impacts our clients’ strategy and business.

For every mention of reducing headcount there seemed to be one or two indications of adding workers (along with some bemoaning of the difficulty in doing so) and the general impression is that the new team members were part of staff expansions rather than refilling vacancies from COVID-19-related furloughs or layoffs.

After investing year after year in digital analytics, our new CMO has made an investment in an additional analyst on the market research side. We look forward to hiring this new person in the next year. This person will focus on our audience of the future (younger and more racially and ethnically diverse).

Researchers offered many expressions of intent to continue providing insights not just data, as a way to deliver more value for internal clients while also demonstrating the limits of relying on gathering data without having staff and processes on hand to make sense of it.

I don’t anticipate any structural changes (hiring, budget adjustments, etc.) but as I get better at demonstrating the value of data and making good use of the tools we have, I think the adoption, acceptance and appetite for data will all increase. I hope that data and my role will have more of a strategic seat at the table early in project planning processes.

There were also multiple comments around organizations becoming more customer-centric, often due to new leadership calling for the change. (You’d like to think focusing on the customer is a given but as we all know, edicts from above are often more powerful drivers than common sense!)

More research is being demanded as internal clients get the message that the new CEO wants VOC justifications for product design/development, etc.

Though sometimes too much C-level involvement/interest is a bad thing.

We have multiple layers of new leadership (from CEO down). Once again insights has a SVP leading the team with a direct line to the overall head of marketing. That alone will get us a ‘seat at the table.’ Unfortunately reorgs will happen. And we are somewhat struggling trying to satisfy these new leaders, some of whom are from different industries. They are demanding changes to tracking studies without understanding all the impacts, only considering their demands. This will make it very tough for some time.

Within many companies and organizations, one welcome (if you can use that term for anything connected to COVID-19) impact of the pandemic was the elevation of the insights function’s internal status as a beacon in the dark, as everyone from the CEO to the regional sales manager suddenly felt tempest-tossed by the economic and societal shifts and changes.

How valued?

While not pegging it to anything related to COVID-19, we asked respondents twin questions on how valued they felt the insights function was at their organizations and in the general business world (using a 0-10 scale from not valued at all to extremely valued).

At the organizational level, a combined 64% rated the value between 7 and 9 and the comments reflected that generally positive assessment.

We have worked hard for a seat at the table and it’s paying off. Market research is a valued business partner.

There is so much demand for insights, we can’t keep up!

The pandemic has changed it for the better.

But alas, the picture is not all rosy in other organizations:

Research is so expensive to execute. There is little knowledge on balancing the practical with the theoretical. It is seductive to half-ass things to keep costs low. This is compounded by the widespread lack of understanding in executing research by our peers. Our internal partners reflect on past experiences where they worked with insights folks who delivered what they needed. What they don’t know is that what they were served was poorly designed garbage a slight step above a gut check.

And others expressed variations of “it depends,” citing factors such a lack of organization-wide belief in insights and a disconnect between management’s words and its actions (sound familiar?) for their middling assessment:

I say 6 because I think our organization is generally pretty interested in research but how much they apply it varies by the individual.

We say we are consumer-driven and consumer insights is our top priority but the company often doesn’t act like that’s true.

Highly valued by the marketing team, not always as valued by our partners across the organization.

As to the reasons why it’s not always valued:

(C)ustomer insights activity is seen as requiring basic skills, a lot of people are teaching you how to do your job, some even try to do it ‘because it’s rather obvious,’ thus insights people are positioned at low levels in the hierarchy.

Too many people just want to go with their gut – that has not worked out well in many cases – but we are a male-dominated organization and have a lot of egos that need to be heard.

Our CX/MR function is very new and we are still trying to demonstrate value.

I was hired six months ago to revamp the MR function at my company. There is great enthusiasm for more and better research yet also a fair amount of reluctance to let go of the way things have been done for decades (without consistent professional leadership in the MR function). I’m content and pleased with the level of enthusiasm for better research, though there are still challenges ahead.

Very valued, but not in the right way. Too much emphasis on scorekeeping, satisfaction research, not enough on insights and supporting strategy. (Stakeholders already think they know what the market wants.)

Similar problems

Respondents seem to feel that research is a bit more valued in the general business world, assigning slightly higher external valuation levels than internal while also citing similar problems.

Research is tricky. When it says what people want to hear they love it. When it gives a different answer they question and challenge the results.

Everyone talks about understanding customer needs. I am not saying MR is the only way to uncover such needs, nor I am saying research does this perfectly, yet research is less valued compared to the value it delivers.

Oversurveying, DIY tools and the widespread availability of data from a host of non-research sources were cited as factors diminishing the impact of insights-gathering.

Marketing research is still undervalued in many companies. A recent focus on data science (a.k.a. data warehouses) is clouding the waters for us before executives. It is a fight to stay relevant and visible.

The democratization of data collection through SaaS tools and the demands for agile research means that a lot of the standards and skills associated with high-quality research are thrown out the window. Also, now it seems everyone is researching the customer. Not unlike the ‘tragedy of the commons’ when we phoned the public, now it is done through many channels. I suspect customers will eventually get tired of being constantly asked for their opinion after every transaction they have with a company.

Everyone thinks they’re a market researcher. The DIY tools place the research capability in untrained hands or in the hands of those interested in lead gen, etc., and it hurts bona fide research efforts.

People are being inundated with feedback requests, forms and surveys. There is starting to be a lot of garbage input.

Look ahead

After assessing the present, we wrapped up the survey by asking them to look ahead and give us their views of the future of the insights profession. Again, optimism generally reigned, with respondents seeing opportunities for personal and professional growth, thanks to research’s pivotal (and long-acknowledged) role as the conduit through which companies hear the voice of the customer.

I think it’ll always be necessary, even if it will constantly be changing and evolving. Our group always says that market research is just one data point alongside many. I don’t believe that will change, even as new types of data points and ways of gathering information come into the picture.

I feel very enthusiastic and optimistic. We’re clearly in a transformation era that is placing shoppers, consumers and people at the heart of business and brand strategies. Data and technology are bigger and bigger enablers but there will still be a crucial need of human intelligence to connect the dots and identify growth opportunities.

The business intelligence movement is highlighting the importance of data insights so research is once again coming to the forefront as important to business decision-making, respondents said.

I’m semi-bullish. If we get stuck thinking about insights as primarily a survey-based profession, we will slowly be replaced. But I’m optimistic that the insights profession will be able to stay current and adapt to a [combination of] more captive data analytics + traditional insights methods.

I feel very positive about the future. The amount of data is exploding every day and the industry needs people to analyze and make sense of it.

If we get the message across that you need dedicated insights people with skills in data and storytelling then the future is good.

The outlook is positive, I think marketing research will be increasingly valued especially for those that can triangulate insights with other non-research data sources (e.g., operational product metrics, financials) to align findings across the business.

Data-quality issues seemed to top the list worries, in the form of fretting over falling response rates and fears about fraudulent panel respondents.

I’d feel better about [the future] if panel sample wasn’t such a problem.

Despite practically all industries increasingly relying on data analysis for decision making, I am becoming skeptical of the quality of insight you are able to obtain via survey research. Too many panels seem to have professional survey takers/task rabbits taking surveys and I think people have become increasingly able to goose quality-detection mechanisms.

Research vendors take note: Despite the impressive progress made in offering alternatives to pages and pages of radio buttons, client-side researchers still hunger for survey tools and approaches that offer respondents a fun, potentially entertaining (or at least not stultifying) experience. 

I think response rates will decline. There’s opportunity in survey research to innovate ways of gathering data that are less time consuming/difficult for respondents.

It will continue to be important but will have to find creative ways to get feedback instead of using long questionnaires.

[S]urveys aren’t short enough, gamified enough to continue to engage humans.

Though, that said, for all of the decades of talk of the need to shorten surveys, questionnaires remain stubbornly lengthy and no matter how engaging a survey platform is, a long survey is still a long survey. Further, researchers are the ones writing (or approving) the questionnaires so if they really want to stop burdening respondents, it’s all within their power to do so.

UX encroaching

A popular topic at industry conferences in recent years has been the intersection of UX and marketing research and the tone has always been very collegial, with the two disciplines sharing many commonalities, but interestingly in this edition of the Q Report survey there were many slightly wary-sounding mentions of UX encroaching on MR’s turf, with the biggest worries seeming to be internal confusion over what each one does and an eventual subsumption of research.

The increase in UX research teams in many companies also seems to be impacting the perception around market research teams, as many companies don’t understand the difference between the two functions.

I think it is heading in a particular direction – UX research and big data. I see a need for my area of expertise waning.

[Research] needs to continue evolving as the boundaries with the other types of insight professions are getting more and more blurred.

Respondents made many mentions of artificial intelligence (AI) and automation and the threats they embody to the marketing researcher’s traditional strong suits of data gathering and data analysis. But commenters also see the rise of dumb data as an opportunity for researchers to step in and give shape and voice to its meaning.

I was worried a few years ago when the rhetoric was that AI and automation would take over our industry. I think two positive things have happened: 1) AI and automation are now focused on enhancing our jobs; and 2) people have realized that a shift to AI and automation left everyone asking ‘why’ consumers or clients do what they do and thus there has been a shift back to qualitative research being critical again.

With actual data available more and more every day, it will be less important to get stated intent data. It is critical that insights and analytics are combined.

We’re moving into much more of a digital space – a lot of the data we used to rely on quantitative surveys for, we can easily get in other ways. I think market research will remain important but will evolve into something much different. That said, there will always be a need for qualitative research.

Along with fears of too much DIY research among internal groups, while there were plenty of expressions of worry over tech-based tools usurping research’s role, whether born of experience using some of those tools (and finding out they’re not going to instantly make research obsolete) or just a general societal trend of being more comfortable with (i.e., less afraid of) technology, many respondents sounded notes of optimism around being able to harness tech for their advantage.

I think the tech bubble of shiny and new will burst once we realize that it isn’t the replacement for traditional research that we all expected. Nothing replaces human interaction when it comes to getting information from humans.

I think a recalibration has been happening – in recent past, there had been a large pendulum swing to big data and analytics away from primary research. I think that things are rebalancing as companies realize that big data can’t solve all ills and yield all insights, and that integrating the two is the best path forward.

And this tech vs. human angle may also extend to the people who work with the data, in the eyes of some respondents, who cited the benefits to marketing researchers of having a long of history of surveying and interviewing people, unlike the (big) data analysts who are more versed in working with data. 

Much of the functional work to conduct marketing research is being automated so the day-to-day tasks of designing surveys, collecting data, tabulating and analyzing data are being greatly reduced. We need to replace those functional tasks with higher-value strategic functions that can’t be replicated with technology. We need to evolve and add more skills to our toolkit than traditional marketing research. Additional skills/experience such as moderator training, data science, organizational psychology, workshop facilitation and others are needed to broaden the value of our function in business and elevate it to a more strategic place inside organizations.

Sounds simple enough!