One of our overall goals with the corporate researcher survey for this year’s edition of the Q Report was to investigate communication – between researchers and internal clients, between researchers and vendors and between researchers’ organizations and their customers.

The value of good communication can’t be overstated. With researchers and internal clients, it’s what helps drive home the value of the insights function. With researchers and vendors, it’s what facilitates a successful project. With organizations and their customers, it’s what lets the customers know their input is listened to and valued.

We also sought to understand: how their insights function fits into their organizations; the tools, technologies or techniques that have had the biggest impact for them recently; and any MR-related changes the researchers foresaw for the coming year.

That’s a lot to cover in one questionnaire and we realize this year’s survey was long. (Hats off to the diligent person who took the time to answer each open-end with the same “This survey is TOO LONG!!!!!” response. Message received!) Part of our reasoning was that in order to give you the most substantive report possible, we needed to ask many questions and perhaps more than the normal number of open-ends. Readers have repeatedly told us that one of their favorite parts of the Q Report is poring over the candid observations of their peers, searching for kindred spirits and similarly tortured souls who are battling – and winning – the same wars as they are. And there’s no better way to add that kind of color to research findings than with some good verbatims.

Do they know MR exists?

We asked researchers to think of times when internal stakeholders didn’t include the insights function in important business decisions and gave them a list of possible answers. At 26 percent, “lack of awareness of the insights function’s capabilities” was the most-cited response, followed by “stakeholders used other internal data to understand the issue” (18 percent), “insights function viewed as too slow” (16 percent) and “stakeholders do not see value in gathering customer feedback” (13 percent).

Of course, there’s no better way to combat a lack of awareness than by improving communication with the colleagues who use your services! We dug a little deeper there with an open-end that asked for more information on the specific areas in which the respondents wanted to boost the quality of internal communication. Upping their data visualization skills was frequently mentioned, as was better reporting in general, along with many mentions of “socializing” the research findings.

“I think we are on the right track. We are investing a larger percentage of our budget on socializing this year and I see it paying off.” 

“People were unaware of our role and what we could be doing to help them. Recently, as I’ve joined the organization it’s been a big focus to get out my team’s name within the organization. People were/are doing a lot of things on their own, results and findings are very silo-ed. We’re trying to be more of a connector of things that are happening.”

“We need to let them know what we can do to help them but it’s a balancing act between giving them a ‘menu’ (which causes them to be prescriptive in what methods we use) vs. telling them ‘come to us for help’ and we’ll decide what methods to use (which is vague and causes them not to know when to come to us for research).”

“I’d most like to improve my ability to communicate with internal departments/customers in terms of helping them think through the business problems they are trying to address, offering more comprehensive insights beyond quant survey and behavioral data, and providing more consultation/direction on next steps following the conclusion of research studies.”

“I’m working now to get better stakeholder feedback after project completion. It’s pretty easy to get, ‘Thanks, this is super-wonderful, you guys rock’ and harder to get really useful feedback on how we can do better, more useful work.”

But despite their best efforts, sometimes researchers aren’t able to make the impact they want. 

“This seems to be more and more of an uphill battle. I don’t remember having to or trying to educate on the importance of research like I have to do here. And receiving no interest in understanding it.”

“Everyone wants insights but it is a challenge to get them to read reports and digest the insights unless they have a specific need. All are so busy. They intend to get to it but don’t.”

“In our organization I think we’re well known for being able to bring the voice of the consumer to a discussion but sometimes they forget to bring us along either because things are moving so fast or they aren’t aware we have a tool to help them.” 

In many cases, it requires the researcher to get out there and do some work, whether it’s selling the value of what they do or learning more about a business unit to better understand its specific needs and how research can help.

“Building deeper relationships – really learning the business, the issues they face. I’ve seen this step skipped a lot which leads to all the above reasons for them to go elsewhere or cut out the research function.”

“More collaboration and less ‘trying to take over the world’ from the customer experience office. It is good to establish yourself as an expert in what voice of customer is but not smart to think you know more than the business line about the actual product or service.”

Thankfully, there were many responses like these, indicating that improving their internal communication was not a big priority because things were already pretty good:

“Our company is great about insights having a seat at the table. I can’t think of any important business decisions where we weren’t included.”

“I don’t have the issue of being left out of important business decisions. My company values the work I do and the perspective I bring.”

Where does MR fit?

To get an idea of how the insights function fits into their organizations, we asked if MR was its own department or part of some other one and generally, the insights function seems to either be on its own (36 percent) or part of marketing (36 percent). A healthy 18 percent said “other,” with “customer experience,” at 6 percent, drawing the next-largest number. As for those “other” responses, some of the department names included: innovation, sales, business intelligence group, corporate strategy, business intelligence and analytics, strategic planning and usability.

What tools are working for you?

Making an impact internally, of course, depends a lot upon the tools you have available to you. Though the traditional stalwarts like PowerPoint and Excel are still depended upon, as are established techniques like focus groups and online research, insights pros have a range of newer options, most fueled by technology, to help them collect and analyze data and disseminate it to their internal clients.

We asked an open-end to gauge which technologies, tools and techniques had made the biggest impact on researchers’ deliverables in the past year. Though we didn’t ask for specific company or product names, many respondents offered them up. The number of references to Qualtrics was eye-opening. Others that received multiple mentions: DiscussIO, ZappiStore, Infotools Harmoni, QlikView, FocusVision Decipher, SPSS, Stata, Tableau, R, Fuel Cycle, Q Research Software, Microsoft Power BI, IBM, QuestionPro, Wincross, Sawtooth Software, icanmakeitbetter, Fulcrum, SurveyGizmo, Digsite, Compellon, FocusVision Kinesis, think-cell, VisionCritical, MarketSight and Medallia. 

“Qualtrics – without it, I would not be able to conduct as many research projects as I currently am. Specifically, Qualtrics Vocalize, which has taken the task of generating reports off my plate on several projects. This year, I’ve also implemented an in-house customer research panel, which will alleviate a lot of vendor interactions and make my department a lot more efficient.”

“InfoScout as a tool has transformed our insights department for measuring household penetration and other key metrics in the convenience channel. Need-state segmentation is also proving to be an amazing tool for us.”

“Fulcrum and SurveyGizmo ... we are able to save SO much money by doing all of the work ourselves!”

Of the tools mentioned, these were all cited multiple times: communities, dashboards, telemetry (passive data collection), agile solutions, eye-tracking, big data, virtual reality, cloud capabilities, blockchain, data visualization, social listening, journey-mapping (“We have a need and execs are seeing the value.”), text analytics (“Helped us summarize a large number of verbatim comments.”) and persona-building.

Some sample quotes:

“Agile, quick research. Everyone wants things fast. Our B2B online community has had success providing insights at the speed of business.”

“Social media listening tools and content analysis have been key components of new projects.”

“High-quality infographics. We are moving more and more into digestible, bite-size learnings/insights.”

“Online insight community – it gives us the flexibility to control our players, our timelines for research projects and the deliverables and objectives of any given project. It gives us control.”

“Social media, of course, because we can reach a wide audience in more broad geographic areas without leaving our desk with very little $$ spent except for $$ spent for respondents.”

“Tableau. Teams across the company just want our survey/research data to create their own deliverables. It has impacted the way we think about AI and automating research reports but we struggle with how to balance that with the insights we need to provide.”

“SPSS & Excel, as always, for analysis. Lynda.com for teaching/training staff. Smartsheet – project management tool – helped vastly with planning for team.” 

“Online community. Implementing the MROC has saved my company probably $300,000, as we can use the MROC for studies we would have had to build out from scratch last year.”

Some non-tech and not-so-new tools got some love as well, with conjoint analysis being mentioned several times.

“Our organization has really embraced conjoint. We used this technique for a project a couple of years ago and the results are being widely disseminated throughout the organization. And people really like what this type of project provides.”

“Texting and mobile phone technology. A generation used to texting seems far less likely to complete ‘traditional’ survey instruments.”

“Availability of great DIY tools (e.g., SurveyGizmo) has enabled lots of simpler work to be done very cost-effectively.”

“Crowdsourcing championed by the CEO. It has been touted as how we develop innovating ideas and that we listen to the ‘crowd.’ It does not really impact what we deliver but sometimes people in the organization think all insights can be done through crowdsourcing.”

“Observational, 1-1 interviews with bio/physio measures including heart rate, facial coding, voice prosody … differentiates pos/neg response between product prototypes where traditional questionnaires show parity.”

“Statistical analysis – we have increased the amount of data we have on our marketing activities and have been able to increase the accuracy of our marketing mix modeling to increase impact on marketing investment decisions.”

“Other software, such as SPSS and Sawtooth, have been invaluable for analysis of our data and performing conjoint and pricing elasticity studies.”

“Interviews and online surveys. Tried and true.”

To get funding to use some of these new tools, you most likely have to make a business case for doing so. We asked respondents to rate the effectiveness of a variety of ways to do that. “Ability to demonstrate ROI” drew the highest percentage of very effective votes, followed by “demonstrating cost savings.” In the effective camp were “vendor information/presentation” and the “ability to demonstrate time savings,” followed closely by “a case study from another firm using the tool or technique” and “a track record of past success with new approaches.”

How well do you know your customers?

To further plumb the workplace cultures, the survey asked how well respondents feel their organizations know their customers. Happily, some variation of “quite well” was the most-offered response. Now, one could say that of course a researcher would give that kind of assessment – customer knowledge is their bread and butter, after all – but along with being keenly aware of what’s going right in the insights-gathering realm they are just as attuned to what’s missing.

“It is the non-customers we fail to understand.” 

“We know them very well – what slows us down is evolving to what our customers want.”

“We have been too focused on thinking of them as consumers, not people. This means we know everything about how they use/choose our products and the category but much less about their full lives. And hence we haven’t connected to them on their key issues. People don’t spend their entire day thinking about our categories!”

“Poorly. Our executives are COMPLETELY out of touch with reality. They think our customers look/act like them.”

“We are a B2B organization and each customer base seems to be department-specific. While some departments might have good knowledge of their own customers and key contacts at customer organizations, there is little transfer of this knowledge or systematic research/info-gathering and sharing across departments, often due to complexity of customer organizations, etc.” 

“We know so little it’s embarrassing.”

“Based on how often we say we know our consumers? Like the back of our hand. How often we really do? They’re like the cousin you only see at the holiday gatherings – we know them a bit, but not nearly as much as we claim to.”

“Pretty well! Sometimes we listen to our squeaky wheels too often but we have people like myself that try and balance that. :) Always room to improve though!”

“We’re in the midst of an organizational transformation, so everyone is trying very hard to know our customers better, which started with identification of target segments. Two year ago, I would say our organization did not know its customers at all. Today, we are making progress.”

Are you happy with the available tools?

So if there are deficiencies in the tools they use to talk to and listen to customers, what are they? We asked if they generally felt the available tools and methods were working for them and, if not, what they would like to see added.

“Listening tools are continually limited. We haven’t seen any reliable tools for sentiment or even text or image analysis that are quick and/or cost effective. As an agency, we move fast, and most research tools are not fast.”

“My sense is that one of the bigger challenges is enabling insights to be disseminated and consumed in a holistic way, so that stakeholders have the most comprehensive view of customers that is possible and that market research is not getting too siloed in its thinking and approach.”

“Many of the tools and techniques still seem tied to approaches that were developed in the ‘nuclear family’ era. Databases from these tools can be like anchors to prevent progress as changes can disrupt the ability to compare new results to old.”

“Some are too broad and don’t gather the context to understand where the customer is coming from. For instance, NPS is an OK measure but it would be even better if we collected or appended data to link the customer’s experience/last touchpoint to understand areas where we are strong or weak.”

“I get the most out of what I have. It’s like that flattened tube of toothpaste – there’s always one more squeeze left.”

“Social listening is tricky – it is useful but ROI is hard to prove. In addition, you get the insights and the marketing team has trouble taking action on the ‘so what’ and to be frank the marketing research teams have trouble helping at times.”

“Survey responses have been going down for years. In that sense they are not effective anymore, we need to open a continuous feedback channel with our customers and not just send them a survey once a year out of the blue.” 

“Panel sampling is getting to be a huge problem. We’ve had two sample companies we’ve relied on taken over by venture capital firms. They seek to boost profits so they can turn around and sell at a higher price. They replace experienced researchers with sales people who know little to nothing about providing good-quality sample. When response rates suck among younger age groups  they throw in river sample without informing us. When results come back completely skewed due to the poor quality of the sample source they can’t figure out what the problem is. When we dig into the data and figure it out they act as if we are being unreasonably picky. These are major companies not small mom-and-pop shops. The implications for the industry are frightening.”

Many responses touched on aspects of a topic that is top-of-mind for most researchers these days: data synthesis. In other words, taking all of those firehoses of information and making them into a manageable trickle instead of a flood.

“I think we have a great range of tools. We have a lot of data from different sources. But we struggle to integrate it all into an understanding of what’s happening in the world of the consumer, and the best way to affect it. We don’t seem to have the time, as we run to complete one study or another, to sit back and think about the whole picture. Internal clients are also frustrated by the output: ‘Study A said this, Study B said that.’ But it’s a manpower issue – who has the luxury of time to think about it all?”

Sometimes it’s not the tools:

“The tools available are fine. It’s our culture and internal politics holding us back.”

“The tools are effective, what we are missing is budget; market research is increasingly expensive and it is more and more difficult to show a clear ROI.”

“It’s a hodgepodge of things. They think they know it all. They sometimes think that if the study is not done by a consultant for $300K then the data is not valid. This is most often for new and young employees that think they know it all.”

“Our tools are effective. However, our end user groups are very low-incidence. Many of our projects are slow because it’s tougher to find end users willing to participate. We have an online community and access to B2B sample but things are not fast enough.”

“Big concern about sampling reliability.”

“Like to develop some methods of better mapping out B2B customer entities that acknowledge the organizational complexities and the various roles.”

“Plenty of tools exist. Our biggest obstacle is that we’re so busy trying to do most of what the business asks us to do that we don’t have much time to go after the issues we probably should be – trying to answer the questions the business WILL ask when they think of them someday. Trying to skate to where the puck is going to be.”

“We operate under a complicated B2B model. Many DIY tools appear to be focused on B2C/mainstream audiences.”

“Listening tools only listen to those that are speaking. Most of our consumers are not out on social media talking about our brands. Our products cost less than $5. The consumers that do write about our brand are not representative. Tools that are REPRESENTATIVE of the marketplace are what are lacking.”

What’s coming next?

If the preceding is a snapshot of many of the issues currently facing researchers, what do they see on the horizon? We asked them to tell us about the biggest MR-related change their organization will make over the next year. Some foresaw having smaller budgets with which to work. There were many mentions of agile research and/or moving toward more agile approaches. Also lots of references to improving or increasing use of data visualization and of automating more processes.

“Focus more on agile research techniques as part of sprint projects where we see a move away from traditional, proven methods to new, faster methods. This is particularly true in new product development where there is extreme pressure to move fast, often at the expense of research/insights.”

“Revising multiple NPS tracker studies (with almost a complete re-do/clean cut vs. the way that the studies have been conducted over the past seven+ years), reducing questionnaire length and setting up processes for more continuous measurement in future years.”

“Looking at newer methodologies and automation as well as knowledge platform (considering new vendors KnowledgeHound, Voxpopme, AYTM, ZappiStore).”

Some looked to the future with excitement, showing the can-do attitude typical of the modern researcher, who always seems to be forced to deliver on some combination of better-faster-cheaper:

“Doing more with less $ – scrappy and agile – early and often.”

“Finding more and better ways to ensure that each dollar spent has the greatest impact on the largest part of the enterprise as possible. Smarter spending for greater reach, usability and impact.”

“Adding new tools. Continuing to do work faster.”

“The integration with other systems so we can research smarter. Cleaning our data from our legacy registration and end user systems and getting the right information into our CRM. Once that is established we will have the foundation for better insights.”

“Some of us ‘traditional marketing researchers’ are broadening our skill sets to also be design researchers as part of experience design teams/initiatives. Our company hasn’t made a position for a full-time, focused XD researcher – yet.”

“We are working towards a better data management system in order to glean insights across multiple sources of data. One company we have looked at and been impressed by is KnowledgeHound but we will need to fight for a big budget increase.”

Some took a more, er, pessimistic view:

“Hopefully we’ll get back to being researchers and not meeting/workshop facilitators.” 

“My company is undecided if market research is a service group or a valued contributor.” 

“We will probably disappear altogether.”

“No idea. I have new bosses, so anything can happen.”

“Our leaders/executives love shiny and new and don’t appreciate what it takes to conduct real research with integrity. It’s a disturbing trend and we’re very worried about the culture here.”

“I would guess another reorganization. They are very popular around here.”

What about the vendors?

The other integral part of the research process, for most client-side researchers anyway, is the research vendor. And effective communication with them is just as crucial as that with internal clients or the users of your organization’s products or services. Happily for both parties involved, communication between vendors and clients seems to be on good footing, based on the responses to our survey. 

There were the satisfied:

“Honestly, we have great vendors and great relationships with them. I can’t think of anything!”

“Communication is currently pretty darn good.”

And there were the snarky:

“Find vendors who can think. That would improve communication.”

“Stop trying to sell and listen!!! Please! And don’t tell me you are innovative! I want real research solutions, not a sales pitch.”

In between those two poles, respondents offered some constructive guidance for their vendor partners on what they are looking for – and not looking for – communication-wise.

“There is a real gap between what our vendors can understand about our business and business challenges and what my internal clients need. That gap creates a challenge when translating our business objectives to a research objective and vendors simply are not good at making that connection. I wish they could understand our needs and drivers better so that they could offer better recommendations from the research. Now, all vendors like to make recommendations but they are either super-obvious and just a regurgitation of the data or simply not useful and thus end up being totally ignored.”

“Sometimes we put out an RFP and we are surprised by what we get back. Some are clearly not doing their research on our company and audience. We even had one that had the wrong organization name. Attention to detail, common sense and easy to work with/responsive are all very important attributes.”

“I would love vendors to understand our business better. Every vendor meeting starts with, ‘So, tell me what you do currently.’ It would be revolutionary for a vendor to come in and say, ‘Here is what I understand about your business and here are some ideas for how our tools can help you.’ To date, all discussions have been me telling them how they can help us (or not).”

“We are sometimes stalled by internal procurement, who doesn’t understand that research is not a commodity. It would help if our vendors could provide materials for corporate purchasing departments that explain that they are reasonably priced but why they are not necessarily the cheapest. I need their help to explain to others what makes their particular shops a better fit for us and why they are worth more.”

“I am bombarded with cold calls and e-mails on a daily basis. It makes it difficult to sift through the options. All communication should include info on the company, a presentation of capabilities and use-cases/white papers.”

Keenly aware

No one study can capture the breadth of the current experience of being a marketing researcher (despite some respondents’ claims that we tried to do just that with our too-long survey!). This time around we focused on communication and found that insights pros are keenly aware of the importance of effective communication and its impact on the key relationships in their daily work.

Taken as a whole, the findings chronicled here show a profession under fire from the usual sources – shortages of time and money, a lack of respect for the discipline as a whole and a deluge of new data sources to wring value from. But just as typical is their perseverance in the face of these obstacles and their ongoing ability to adapt their systems and methods to rise above them and continue the search for business-changing insights. 

Where does the marketing research/insights function fit? 

New teams

“I'm the first official ‘research’ person to work for the company.”

“When I was hired two years ago, I was the first consumer insights person, and at that time I was under the ‘strategic analytics’ department. When we hired a second researcher about a year ago we became our own consumer insights department.”

Reporting directly to the top

“Insights reports directly to COO. Remains highly cross-functional with marketing, product, sales and customer success.”

“Department has been set up by me. It used to be part of marketing, now it is part of the analytics dept. reporting directly to CEO.”

“Our function is currently part of the strategic planning team, who reports to the CEO. Over the years, we’ve been in various departments – communications, technology, marketing – but have always had the same MR function.”

Its own department

“Marketing is separate. We are in our own research area (for good reason).”

“It has always been its own department but where it sits overall has varied. Originally it reported to the VP of new product development. Currently it reports to the VP of marketing and product management.” 

Still figuring it out

“Our team has moved all around. Where it's situated at this point honestly makes no sense. We're within the analytics group, which is within the loyalty group, which is within the marketing group.”

“All the research and data analysis departments at my agency have been in flux for years. The company is still trying to figure out how to organize the groups. There is not enough cross-training and too many silos.”

Don’t know!

“Don't know. Have never been in touch with marketing at my company. You do make me curious because of this survey.”

Methodology

The Q Report work life study of corporate researchers is based on data gathered from an invite-only online survey sent to pre-qualified marketing research end-client subscribers of Quirk’s and members of ESOMAR. The survey was fielded from June 6 to June 28, 2018. In total we received 1,075 usable qualified responses. An interval (margin of error) of 2.9 at the 95 percent confidence level was achieved. (Not all respondents answered all questions.)