Editor's note: Laura Cusumano is a marketing research consultant based in St. Louis.

An age-old query when commissioning research is, does the marketing research agency have experience in the category? Every client company asks a prospective agency that question. It’s an obvious, straightforward one. Client-siders spend significant amounts of time reviewing and discussing the specifics of the prospective supplier’s experience. But how critical is experience in a category to effectively executing a research project? Perhaps the more germane question is: Do you have the skills and experience to successfully execute this research in a timely, cost-effective manner? 

For a research agency to effectively deliver, direct category experience is only one aspect of its overall ability. For example, if an agency has only 10 years of collective experience among the research team in three categories across five clients,then direct category experience is probably very important for a successful project outcome. If the agency has 50+ years of collective experience in the team in 25+ categories across 25+ clients, then direct category experience is nice to have but probably not a requirement for success.  

“Do you have experience in…” also demands an inquiry about when the projects were completed. If an agency completed 10 projects three years ago, are they more or less qualified than an agency that has completed three studies in the past year? Another issue is the activity of a category. Some categories are stagnant while others are dynamic,with an ever-changing landscape and market players.

Answers to “do you have experience in…” also depend on the degree and extent of the experience. Small, one-off studies probably do not provide the same level of experience that comes from a series of large, iterative studies. Is the experience in five small telephone projects equivalent to two, multi-wave tracking studies?

An accurate assessment of the specific value of “category experience” for selecting an agency would require a conjoint exercise that measures all the variables possibly considered, both consciously and unconsciously, when awarding a research project. These include but are not limited to:

  • years of marketing research experience;
  • years of related experience: marketing, sales, clinical or client-side;
  • total number of categories worked in;
  • similar categories worked in;
  • diversity of categories with experience;
  • the client’s previous experience with the research agency;
  • the scope of the project vs. the core competencies of the agency;
  • the ability to successfully recruit target audiences;
  • the marketing research approach and techniques under consideration;
  • the ability to meet the project’s time frame and deadline;
  • the number of years the agency has been in business;
  • the size and makeup of the team who will execute the project;
  • the project director leading the project;
  • the physical location of the research agency; and
  • cost and fees.

It is not likely that we will ever have the utility scores for these attributes but we can still address the question of the experience level required to successfully execute a study. 

As you have guessed already, the answer is: Experience is required in some but not all studies. So, the relevant question is, “In what studies is experience required?” That, naturally, mainly depends on the category, the type of research and the respondent type.   

Taint a study

It is easier to address when experience is NOT really needed and can actually taint a research study. 

Experience may not be needed in studies that are considered straightforward, are shorter, more directive and are in less-complex or newer categories. Using a skilled, senior-level agency with little or no experience in a category can be quite valuable in exploratory research, which needs an open mind with no preconceived notions. 

Sometimes a brand team needs to hear that the “emperor has no clothes” and it is harder for a marketing research agency entrenched in the brand or category to be able to step away and be totally objective. A naïve agency goes in more open-minded and can expand the view without drowning in the details and history. Generally, a new agency has not “drunk the Kool-Aid” and can challenge established beliefs or expose new, valuable morsels. However, if a research agency without direct experience is selected, it may require more time and effort for the client and adds a level of risk to the research. 

Assuming that experience is always necessary can bias or limit the research and its value. But determining if experience is necessary is not easy and needs to be analyzed and considered carefully.