The Wild West of the market research industry: The history of marketing and the future of insights
Editor’s note: Brooke Reavey is associate professor of marketing at Dominican University, River Forest, Ill.
Marketing, as an academic discipline and profession, has seen better days. The market research industry has seen numerous innovations and technological advances over the past decade(s). Some have been great for the industry and some, in my opinion, worse. The industry has an increasing issue with sample quality, sample fraud and educating the industry on current data analysis methods. The question remains: Will marketing continue to operate in the Wild West like it has for decades, or will public policy regulations help save our industry and, more importantly, our expertise?
Why we should care about the history of marketing
Marketing is still a burgeoning business function. It is only a little over 100 years old, which in comparison to the other business disciplines (i.e., economics and accounting) is still very young. Marketing was initially recognized as an essential business function when locomotives started moving goods like wheat and coal long distances. Economists realized that a surplus in supply does not always decrease prices because high demand was elsewhere. The agriculture markets set the stage for where we are today. At the time, marketing was seen as a disruptive discipline identifying how and where goods were dispersed. The modern digitized era where we focus on who and why would not be possible if we did not have public policy oversight from the early 20th century.
Each era of marketing has seen its own set of laws and regulations. For instance, in the early 1900s when marketing was recognized as a distribution function, we saw laws that helped standardize food preparation and grade food items. For example, before laws that demanded for the pasteurizing of milk and transporting it in sterilized containers, consuming milk in cities was one of the leading causes of mortality for young children. Additionally, the USDA implemented grading systems for items like fruit, establishing a pricing scheme proportionate with quality. For example, we typically purchase U.S. Extra Fancy apples in the grocery store and Grade A or B apples are used for making applesauce.
Later in the 1940s, post-WWII, as the competition among manufacturers increased, we began seeing more laws regulating advertising regarding trademark infringement and false advertising. However, the zeitgeist of our modern marketing era surrounds consumer data privacy. That begs the question – who owns the data?
Do you own your data? Does the app that lets you use their service for free own your data? Does the company that consolidated and brokered your data to a third party (along with 250 people just like you) own it?
Data privacy laws such as GDPR and CCPA and burgeoning laws such as the EU-US Data Privacy Framework and the federal-level American Data Privacy and Protection Act (ADPPA) are set to help us navigate these muddy waters. Nevertheless, Section 230 of the Communications Decency Act is one area that has gone unnoticed and could potentially influence the market research industry if and when it is repealed.
The emergence of Section 230 and DIY market researchers
As the marketing research industry has evolved, one area that has quietly gained momentum is DIYers who use survey software platforms (i.e., QuestionPro, Qualtrics and SurveyMonkey) to occasionally collect data. Some collect more data, and some collect less data, but regardless, there is virtually no oversight regarding survey creation (i.e., the types of questions asked), data storage or data analysis. This is in part due to Section 230 of the Communications Decency Act.
Section 230, as it is commonly referred to, is a section of Title 47 of the United States Communications Decency Act, that – for the most part – indemnifies website platforms concerning third-party content creation. For instance, Section 230 is one of the reasons why Yelp.com and RateMyProfessor.com are allowed to publish negative reviews without any fear of being sued for libel because the platform is indemnified for posting content that another user creates. Companies often use survey platforms instead of using a marketing research firm for cost savings, faster access to the results and inherent distrust in third-party vendors. The key point, though, is that there are no associations such as the Insights Association, ESOMAR or the university’s IRB/ethics board that oversee the practices of DIY market researchers. Consumers can’t call any government agency to report an invasive question on a survey (i.e., SSN, driver’s license number, etc.), an unpaid incentive they were promised or have any confidence that their PII data is stored separately from their responses.
How often are HIPAA violations occurring? How many surveys turn into sales pitches? Currently, Section 230 shields DIY platforms from regulating the content their customers create, but can’t we all agree that this creates a bigger problem for the professionalization and trustworthiness of the industry?
How big of a slice do DIY firms have anyway?
Identifying who uses DIY platforms helps us understand why they use them. The main difference between DIY platforms and traditional vendor market research firms is that DIYers are self-taught and take guidance from the software platform for training. For instance, HubSpot has a five-minute video explaining that researchers should conduct qualitative interviews with cart abandoners when developing buyer personas. These training videos make market research techniques, like an IDI, appear very easy to conduct and can trigger hubris in the DIYer. While the training videos were not created to replace training – like a RIVA Master Moderator certification – the videos do not mention that one might need more planning or training to conduct a qualitative interview. Another example is the pre-programmed surveys. There are often no sampling guidance or suggestions for advanced analytics. In my research, I discovered that most DIYers use frequencies as a basis for their decision-making because that’s what the platforms report.
In comparison, traditional vendor marketing research firms typically have specially trained experts doing the work. Experts’ knowledge transfer is often guided by higher education and/or experience through continued education, on-the-job training or a university certificate program. The key differences are where the researchers obtain their knowledge and the knowledge transfer of the insights.
According to the Insights Association’s Insights and Analytics Report, in 2021 DIY survey platforms accounted for 2% ($1.4 billion USD) of the insights industry’s market share. Notably, in 2021 ESOMAR’s Buyers and User Study discovered that approximately 50% of brands stated that they are switching over to in-house data collection with (one can assume) DIY platforms. DIY platforms will continue to accelerate in growth.
Of more consequential concern, though, is defining what constitutes a DIY platform. Using the categories created by the Insights Association’s Insights and Analytics report (Table 1), digital data analytics platforms and DIY survey platforms should be combined. My research discovered that more than 80% of digital marketers use survey platforms as a portion of their job function. Survey platforms are not the only tool people use to collect customer feedback. Many social media and digital marketing platforms offer similar technology when seeking feedback from customers. These platforms are creating their own training and certification systems that prepare modern market researchers. DIYers also frequently turn to YouTube for help. I believe the true portion of DIY platforms is closer to 30% ($18.5 billion USD) of the marketplace than the 2% stated in the report.
Source: https://www.insightsassociation.org/Resources/Insights-Analytics-Market-Report
Navigating the next century of marketing and marketing research
Marketing has evolved into a 24/7 digitized workhorse. Public policy regulations have helped shape marketing to where it is today by eliminating nefarious business practices and ensuring consumer safety. MarTech has helped the market research industry grow in innumerable ways. However, by allowing the entire marketing industry to grow as quickly as it has and with (essentially) no oversight, many market research firms will be left scrambling when Section 230 is repealed. As an industry, market researchers need to band together to decide the next steps. Determining what the professionalization of our industry looks like should be top of mind as we navigate the next century of our profession.