Tell us a little bit about yourself

Editor's note: Ted Pulsifer is executive vice president of research firm Schlesinger Group.

Respondent opinions and experiences are the drivers of strategic business decisions. Some suppliers, more than others, have invested in strategies that support the recognized need for a wide and diverse pool of research participants who trust the process and are engaged. Winning respondent trust is not easy; consumers are learning the value of their data and hold companies that deal in their privacy accountable. Panelists are entitled to know what will become of their personal data. Greater transparency between sample providers and participants has become a vital part of the equation. 

The respondent experience also remains a principal focus for our industry. The most successful providers have been relentlessly pursuing innovative ways to shorten surveys and improve engagement to ensure respondents trust and enjoy the panel membership and the survey process. This includes guiding survey designers towards doing their part to create positive survey experiences that will drive trust with consumers.

Technology has spawned numerous segments among the general population based on how they want to engage with surveys. New solutions have allowed for more personalized experiences for a wide variety of respondents. It is a two-way street – both research companies and panel owners are adapting to the use of automation and machine intelligence to attract, manage and retain respondents. Sample providers are upgrading their portals to ensure a seamless experience across devices (smartphones, tablets, desktops). It doesn’t end there. Everything from matching respondents with the most suitable surveys to incentive management is becoming automated. Leveraging technology for human-centric interaction and engagement will deliver an augmented respondent experience.

The online sampling industry has made major strides in recent years. The most noteworthy has been the automation of project specifications and quota communication. Programmatic sampling is helping research providers not just run surveys faster and at a lower cost but also improve data quality. With deeply profiled panel data shared via APIs, both sample buyers and sellers are aligned on the benefits for survey outcomes, minimizing answer bias and reducing respondent dropouts. Soon, the programmatic approach will also permeate other steps in the sample delivery value chain, including continuous profiling at deeper levels (emotional, psychographic, etc.), feasibility assessment, accessing hard-to-reach audiences and incentive management.

On a new track

The COVID-19 pandemic has put our lives on a new track. The most significant and irreversible change has been the accelerated shift to digital, with adoption and increased use of online accounts and researching and buying more items online. There is an increased focus on health and wellbeing and on financial health. There are changes in when an individual will want to take a survey; changes in their profile information; changes to topics, products and services they will want to engage on and the channels they use to engage with us. Brands are already noticing the shift in their consumer sets and at the same time, panel owners need to be continually engaging with panel members for profile updates and testing recruiting and engagement methods and channels.

Schlesinger Group recently polled 18,000 panel members across the nine countries – U.S., U.K., Canada, Germany, France, Italy, Spain, China and Brazil. Forty-five percent of respondents have been taking more surveys during the pandemic vs. 13% who had less time for participating in surveys; 91% of respondents shared they are participating in the panel engagement activities such as contests, polls and quizzes. Unexpectedly, use of mobile devices for taking surveys had also increased for 37% of respondents, despite most people staying home.

In a world of markets experiencing massive disruption and evolution, gathering reliable and compelling data for deeper insights is paramount for brand success. Choosing your partner for sample delivered at the quality, speed and scale you need nationally or across the globe is a key factor in gathering actionable data for a valuable return on research investments. Understanding the trends that influence quality, speed and reliability will help you ensure your decision criteria are relevant to overcome today’s research challenges.

Daunting process

Choosing a panel partner can be a daunting process. Panel owners vary by their industry, panel demographics, respondent quality, incentive mechanisms, panel management, etc. This short guide provides you with a framework for a primary focus, along with key considerations to help you identify the best panel partner for your objectives.

Understanding panel composition for representative sample and on-target types

Explore panel size and diversity 

Panel strength is the key determining factor for faster time to market and high-quality data. As a researcher, you should know the panel size and diversity in your countries of research. For example, if you are aware that a panel composition is more skewed towards males or has lower representation from a particular geography or income level, you would have to normalize the data to adjust for those differences. This level of information is also important to understand your potential reach to nationwide and hard-to-reach audiences. Depending on the audience of your surveys, explore the B2B and B2C panel strengths individually.

  • Does your panel partner cover enough countries to contribute significantly to your project mix?
  • For your target countries, does the provider have a sufficient sample population with online access?
  • For B2C panels, does the provider have sufficient respondents across age, gender, household income, education, location and job title?
  • For B2B panels, can the provider access your specific employment status and the organization’s employee strength?

Ask about the panel recruitment methodologies 

As a research buyer, you expect a diverse panel that provides the least possible bias. You require a pool of respondents who are not just interested in the incentive but also in sharing their opinions. The key differentiator among panel owners is their recruitment method; some are still using traditional channels, while others have expanded their outreach via a multipronged strategy. In addition to the traditional methods such as e-mail, telephone and referrals, the newer approaches, collectively known as river recruiting (affiliates, social media, online ads, gaming apps), help onboard people across a wide range of demographics, behaviors, and preferences.

  • What channels does the provider deploy for recruitment? (You are looking for a mix of methods to ensure diversity in demographics and of interests and behaviors.) 
  • Did the provider custom-build the panel or did they purchase an existing panel? 
  • Does any part of the panel comprise a purchased list? And, if so, do they keep these members as a separate panel?

Assess the depth of panelist profile data

Owing to brand advertisements that create more choice, people are clustering around their micropreferences. And those cluster members are in flux as people continuously change preferences. So, it’s important to assess both the depth of panelist profiles and the frequency of updates. Mapping panelist preferences is the key to finding the best-fit panelists for your surveys. Most panel owners use onboarding questionnaires for understanding their panelist lifestyle and preferences – anything from their favorite footwear to sustainable buying to hobbies and interests. Panelist profile data enriches survey responses by allowing a granular understanding of your market. We recommend you explore:

  • How many consumer, B2B and health care profiles are collected in the database?
  • What type of profiling questions are asked and how often are they updated?
  • What type of information is collected on the panelists (demographic, psychographic, firmographic, etc.)?
  • In addition to the onboarding questionnaire, what mechanisms are used for ongoing deep profiling?
  • Can panelist profile information be shared in real time?

Assurances of data quality

Understand the processes for rigorous verification 

As a researcher, how can you be assured panelists are truly who they say they are? Although panel owners and independent technology firms have developed a plethora of tools and technologies for panelist vetting, the effectiveness can vary. It is crucial to understand how your panel partner verifies panel candidates and monitors their in-survey behavior. Some panel owners use technologies from Imperium, Maxmind and Sample Chain to verify panelists. Others develop data science-based tools to verify panelists and map their survey activity across devices, browsers and geographic locations.

  • Which third-party partners are used to verify panelists?
  • What in-house processes are deployed for identity verification, fraudster prevention, cross-panel de-duplication and redirect-fraud prevention?
  • Is survey behavior mapped to eliminate fraudulent panelists and undesirable behaviors?
  • Is a respondent quality-tracking system in place?

Ensure there is meaningful engagement by panelists

With a multitude of attention-grabbing entertainment platforms in the marketplace, considerable effort is required to keep panelists engaged. Here are the three ways panel owners keep respondents coming back to take surveys and share authentic survey responses.

Respondent incentives. Learn about the incentivization programs in place. Does your panel partner have a wide range of rewards to match the needs of diverse panel groups? What may be attractive for a 45-year-old female in the suburbs of Chicago may not hold the same value for a 24-year-old male in downtown San Francisco. Choice, speed of delivery and customization are three essential levers in creating respondent incentive satisfaction. Data science and process automation are the key to aligning incentives and delivering them promptly. Ensure that your panel partner has effective and streamlined processes for incentive management.

  • What type of incentives are available for panelists?
  • Is there a system for incentive choice?
  • How is an appropriate incentive determined?
  • How are incentives delivered?

Panelist motivation. Inquire about unique ways in which your panel partner motivates their panels and communities. Leading panel communities are customized around member demographics and interests. If the panel comprises both B2B and B2C members, are the messaging and conversations customized for them? Also, explore how actively the panel partner is communicating with them on social media. As a good practice, panel companies run frequent contests, polls, quizzes, games, etc., to engage their panelists.

  • How frequent are panelist engagement programs?
  • Is there communication with panelists outside of research interactions?
  • On which channels are panel communities most active?
  • What is the mix of long-term panelists vs. short-term panelists?
  • What is the mix of high-frequency vs. low-frequency survey participants?

Panel and survey experience. It is indubitable that a positive survey experience leads to better survey outcomes, completion rates and future participation and to the good standing of marketing research. Survey platforms should be appropriately branded to build trust and present a purpose. Some providers go as far as to create portal personalization. Given the range of online user behaviors, it is important to evaluate the user experience across major device formats.

  • Does their panel portal have a unique brand identity?
  • Is the brand identity consistent across all touchpoints such as e-mail, social media, incentive delivery, etc.?
  • Does the panel owner provide an optimized user experience across device formats (mobile, desktop, tablets)?
  • Does the provider give guidance around survey questionnaire design and length and design concerns that may impede the survey experience?

What will you get? The sample delivery

Speed of delivery: explore programmatic sampling capabilities. With the programmatically delivered sample, you can field surveys faster, create client reports more easily and reduce human error, all while minimizing sampling costs. No one panel owner can provide the sample needed to reach every type of global participant. To address this, large sample marketplaces have made programmatic buying and selling of sample mainstream but as a responsible research buyer, you will wish to evaluate your panel partner’s capability to deliver sample seamlessly via APIs – the software intermediaries that allow multiple panel applications to talk to each other.

  • Does the provider leverage APIs to connect you with leading panels to deliver sample?
  • Can their panel portal be easily integrated with your research platform?

Participation quality: request sample performance data. Request metrics that will allow you to feel confident in sample performance. Ask your panel partner for data on: reconciliation rate (percentage of completed interviews that were reconciled to being not complete); number of active respondents in the last 90 days; total completes in the last month and the last 12 months; average number of unique panelists visiting the panel portal daily; annual panel attrition rate; percentage of attrition that is replenished annually.

Operational reliability: ask about expertise and processes. No matter the advancements in programmatic technology, seamless delivery of projects and high-quality outcomes require an experienced team for bidding and project management, along with operating to effective processes. Evaluate how your panel partner is adhering to in-house processes and is compliant with industry codes and international regulations. If your panels originate from other countries, keep a checklist of regulations that you and your panel owners need to meet.

  • How many years of industry experience does your provider have?
  • What is the average time of bid turnaround?
  • What is the average time to field?
  • How does your provider manage compliance around GDPR, COPPA, HIPPA, FERPA, etc.?
  • Who is accountable for compliance and how is it monitored?

And how will you feel when you have got it? 

The experience. No matter the advancements in technology-driven sampling, it takes talented people who are deeply customer-centric to deliver consistent experiences and outcomes that take relationships beyond the transactional to form trusted partnerships. Getting the sample and data you need should be the minimum you should expect. Ask yourself what added value you seek, what you require that goes beyond a project success, what will help you grow, what allows you to sleep at night and what proves your partner has an uncompromising commitment to get you there. Start by asking:

  • What are your guiding principles for your customer experience?
  • What is the ratio of the bid team to the project management team?
  • What is your escalation procedure?
  • How does the provider track and act on their customer experience performance?
  • How are preferred partnership agreements established?

A telling indication

Being curious is key to mapping your path to achieve exceptional research outcomes. The extent to which your provider welcomes your questions will be a telling indication of the quality of your future partnership. Your provider should have impressive and reassuring answers to your questions, which focus on panel quality and diversity, minimizing sample bias, recruitment and verification capabilities and the programmatic technology to manage quality and speed across multiple sources.

The above questions are best suited for the initial assessment of the panel partner. An additional in-depth resource for your evaluation process is a sample provider’s answers to ESOMAR’s 28 Questions to Help Buyers of Online Samples.