It's mutually beneficial

Editor's note: Gerry Katz is executive vice president at Applied Marketing Science Inc., a Waltham, Mass., research firm.

A few years ago, I was approached by Ralph Oliva, the executive director of the Institute for the Study of Business Markets (ISBM) at Penn State University. The ISBM is a consortium of nearly 100 business-to-business (B2B) companies that meet several times a year to address some of the unique problems of B2B companies. He wanted me and Professor Abbie Griffin to put together a two-day general-overview market research course specifically for non-researchers at these types of companies.

This sounded like fun! But what particularly caught my fancy was his request that we include a unit on how to be a good client (i.e., how to be a smart purchaser of market research services). Most consumer products companies already have a formal market research function staffed by experienced market research professionals. But many B2B firms do not. And when it comes to the development of new products and services, where extensive market research is usually required, that decision will often be made by a project team leader who may have little if any experience in dealing with market research vendors.

While Oliva’s request was certainly sensible, for me this was almost cathartic! I’ve spent just about my entire near-40-year career on the vendor side. And for anyone who has ever worked on this side, we all know the things that our clients do that make us crazy and too often detract from the quality of the final product. Much has been written about how to be a good vendor and what pleases clients. But here was my chance to tell a group of potential clients how I wish they would behave, something I’ve almost never had the nerve to tell them to their face!

So, as requested, Professor Griffin and I put together a unit in our course that addressed all of these issues. Some members of that first class felt that we were being entirely too self-serving – to which I plead guilty! Since then, we’ve refined the unit and delivered it several more times and I am more convinced than ever that we are doing these potential clients a helpful service. In essence, we are trying to teach them how to get the most out of their market research initiatives. But really, isn’t it just as relevant for experienced purchasers of market research services to brush up on their research-buying skills?

So here goes. Here are some of the things I’ve always wanted to say to my clients but rarely had the opportunity – or the guts! (I’ll bet that there are some fellow vendors reading this who are already salivating!)

When to outsource

First, it needs to be said that it is not always necessary to engage an outside market research supplier. With a little training and experience, there are certain things that you can do yourselves, saving thousands of dollars and hearing things firsthand. For instance, you can learn to conduct your own one-on-one qualitative interviews and, with some of the new authoring tools such as Zoomerang or Survey Monkey, you can even do simple quantitative surveys on your own.

But there are plenty of good reasons or circumstances to outsource to a professional:

Resource constraints. Conducting even as few as 10 to 15 qualitative interviews requires quite a bit of people time. If you and your staff don’t have that kind of time, you can always hire it.

The need for confidentiality. In most cases, if you are conducting research yourselves, you’ll need to identify your company as the sponsor of the research. In some cases, however, you might want to maintain confidentiality so as not to risk revealing to your competitors the things you are working on. An independent market research vendor does not need to reveal the name of the study sponsor.

The need for expertise that goes beyond your own capability. While you may be able to do the simple things yourselves, many types of market research require a level of expertise that simply goes beyond what you already have or could easily obtain. For instance, focus group moderating requires a good bit of experience and any kind of quantitative technique beyond simple cross-tabulation usually requires some knowledge of database management, statistics and multivariate techniques.

The need for speed. In almost every case, an outside market research vendor can get the job done faster than you can doing it yourself. Why? Because it’s what they do – full time, every day – not just one of a dozen or more projects you need to juggle just to keep your head above water.

High-visibility projects. When a particular project takes on high visibility within the company, an outside vendor may simply have more credibility. Even though they might draw the same conclusion as you would had you done the study yourselves, there is certainly something to be said for an outsider’s objectivity.

What to outsource

Even if you do decide to conduct market research on your own, there are certain activities that may still make sense to outsource. For instance:

Respondent recruiting. Whether you are using an online database or recruiting people by telephone, this is often a difficult and thankless task. It is very similar to prospecting for a sales organization and it’s not for everyone. It requires a certain level of telephone skill and organizational understanding to daisy-chain your way through a company to find just the right type of person.

Off-site market research facilities. Even if you’re going to moderate the interviews yourselves, it often makes sense to use a professional market research facility. These are well-equipped for viewing, recording and managing the respondents. They can easily greet them, feed them and handle the honoraria, even if you’re doing the interviewing.

Transcription. If you want to have a verbatim transcript of a qualitative interview, be forewarned: do not give this task to one of your own secretaries or administrative assistants. Transcription is a specialized skill that requires specialized equipment. Every city has dozens of transcription services that you can locate either online or in the Yellow Pages. They are typically not all that expensive and can provide a fast turnaround when needed.

Sophisticated quantitative studies. As alluded to earlier, unless you have someone with just the right skill set and experience, you’ll want to outsource these types of studies in almost every situation. Some examples here might be studies having to do with pricing, demand forecasting or causal modeling.

How to work with market research vendors

One of the precepts of market research that I learned very early in my career was not to bother doing a study if you are already sure you know the answer. Human beings have a way of hearing what they want to hear, and if you already think you know, it requires a great deal of intellectual honesty with oneself to hear something different.

First, you’ll need to put together a project plan that answers three simple questions: What is it we want to learn? Why do we need to learn it? What will we do with the information if we get?

This sounds elementary, but I am amazed at how often clients don’t bother to make these things explicit. This usually results in tremendous scope-creep (i.e., a highly unfocused study that attempts to answer a million different questions but fails to answer any of them particularly well). A smart vendor will help you to crystallize your thinking here, but if you keep changing your mind (which happens far too often), it is maddening for us and hurts the quality of the finished product.

Another thing you’ll want to give your vendor is an unambiguous and available point of contact. Sometimes, it is unclear to the vendor who they should be talking to on the client side. This puts us in the uncomfortable position of having to guess about your organizational hierarchy and politics. It is the client’s job to work these things out behind the scenes and to clarify them at the start of the engagement. You need to tell your vendor who should be talking to whom and about what kinds of issues.

Often, there are multiple levels on both sides, each with different points of contact. For instance, the vendor’s account executive may deal primarily with the project team leader, while the vendor’s field director might need to deal primarily with someone in IT who is responsible for generating a list of customers from which to recruit prospective respondents. Just be clear about how you want it to work.

One other consideration: Choose people who are likely to be available and reachable on short notice. If your primary point of contact is always traveling to Asia or is frequently tied up in all-day meetings, maybe you should assign someone else, or at the very least, have a backup person who can speak for the organization. During the course of a project, things come up that require a quick decision – should we adjust the quotas for the sample design; should we reword a question; should we increase the incentive, etc.? We can’t wait until next week to decide some of these things without losing momentum and impacting the schedule!

One other comment is in order here. It is strongly recommended that your primary point of contact avoid defining their role as that of a gatekeeper (i.e., the person who tries to control all of the communication during the project). Early in my career when most of my clients were consumer packaged goods manufacturers, there was a protocol that I detested. Outside market research vendors were told that they could only speak with people in the client’s market research organization and that they were never to speak directly to a brand manager without the permission and presence of that market researcher. The logic was that, too often, the brand managers (who had a reputation for being highly aggressive) would try to manipulate the study in order to influence the results.

While I understand the logic of this protocol, it always seemed pretty silly to me. Most outside vendors are smart enough and strong enough to be able to push back and to make sure that nothing is agreed to without the knowledge, participation and agreement of the client’s market researcher. And whenever I was called by a brand manager, it was standard practice to call the market researcher the minute that call ended to make sure that they knew what had been discussed.

With today’s reliance on e-mail, it’s become a lot easier to keep everyone in the loop, so there is less of a need to control communication. We usually like to err on the side of overcommunicating; that is, we ask our clients to copy our entire project team on virtually every e-mail or other kind of communication and vice versa.

The RFP process

Even companies who are not very experienced with market research usually require a formal request for proposals (RFP) process or some other process for obtaining multiple bids. There are at least three good reasons to do this: to make sure you are getting a fair price; to verify that you are hiring a quality vendor; to obtain assistance with project design.

We vendors hate the RFP process but certainly understand its necessity. Why do we hate it? It goes well beyond the need to “sharpen our pencils” on the price. We all know that even when a client is obligated to obtain competitive bids, they often have a preferred vendor who, deep in their hearts, they would like to hire. Salespeople refer to these as “wired” RFPs. Of course, we never know whether this is the case or not. So, we often put in days of effort preparing a detailed bid and proposal, when there is really almost no chance that we will win the project regardless of our price or the creativity of our design. What often happens is that the client develops a checklist of criteria with the preferred vendor in mind such that they will get a check mark on each and every one of these criteria while almost no one else will. The rest of us all become “column fodder”!

The thing we hate even more, however, is when an RFP is overspecified, with the client specifying the types of questions they want answered and the exact methodology with which they want to do it. In these cases, our role becomes that of a commodity provider, where everyone is required to bid exactly to the specifications. (This is often a clue that the RFP is already wired to a particular vendor.) When you hire a professional outside market research firm, a big part of what you are paying for is the creativity that they bring to the process in how best to go about answering your questions. Most RFPs will tell you that it is fine to depart from the given specifications but in my experience, if you do, you will usually lose.

Another question that clients need to think about has to do with the sequence of the bidding process. In most cases, the RFP will be sent out to five to 10 possible vendors and perhaps half of them will respond with a proposal. The client then reads through these and selects two or three finalists. The finalists are then invited to come in and present their capabilities and their proposal and then a final decision is made. But I’ve found that the smartest companies often reverse the process – they invite the five to 10 vendors in to discuss their project before they even write the RFP. Certainly this requires more time on the part of the client. But the advantage is that they get to meet the people, listen to the most creative methodological ideas from all of the vendors and to refine their thinking as they go along. Then, when they do put together the RFP, it is usually a more informed document that allows for several different approaches to attacking the problem.

What you should expect from a vendor

When you do choose to outsource a project to a professional market research firm, there are a number of things that you should expect and insist upon:

Integrity. Many research industry societies publish standards that their members must agree to. Among these are the Council of American Survey Research Organizations, the Marketing Research Association and ESOMAR. Of course, membership in these organizations doesn’t guarantee perfect ethical behavior but it’s usually a good sign that they take these things seriously. These standards cover things like avoiding professional respondents, ensuring data integrity and not engaging in shady practices like sugging or “selling under the guise” of market research.

Timeliness. Good market research firms emphasize project management. While there are many legitimate reasons why a project schedule might slip, good firms deliver when promised most of the time.

Intelligence. If you do hire an outside vendor, there will probably be one or more occasions when you need them to present results to your peers and your upper management. You should expect them to be well-prepared, to have good presentation skills, to handle questions effectively and to speak intelligently during these meetings.

Insight. Almost any market researcher can stand at the front of the room and show you tables or graphs. But that’s not enough. In our company, we make it a practice to always ask ourselves “So what?” for every table and every graph. The market researcher’s job is to tell you about the numbers and what they mean

What a vendor should expect from you

Of course, this relationship should never be viewed as a one-way street. There are things that a vendor has a right to ask of you, things that make the project go smoother and better. These include:

Timeliness. This one goes both ways. While you have a right to expect timeliness from your vendor, there are certain things during the course of a project that require sign-offs from you. For instance, you may need to review materials such as screening questionnaires and discussion guides, not to mention early drafts of the various reports. To keep to an agreed-upon schedule, the client needs to move these things along efficiently.

Flexibility, fairness and realism. During the course of a project, things sometimes don’t happen the way that either the client or the vendor expects. For instance, it is pretty standard to overrecruit for qualitative research to allow for the fact that some people don’t show up. As an example, if our sampling plan calls for 30 one-on-one interviews, we’ll usually recruit about 36. Sometimes more than 30 actually show up, in which case our client gets a little bonus. But sometimes the no-show rate is higher than expected and we’re only able to complete, say, 28 interviews. Technically, you have a right to insist on 30 completes. But will the study be lower in quality without those last two people? And conversely, will it be appreciably better with three or four extra respondents? Probably not. And is it worth delaying the schedule in order to get those two extra interviews? Maybe – and maybe not. But it is extremely frustrating to vendors to have to deal with this kind of inflexibility when we know we’ve given it our best effort.

Clarity. You’re the customer and so we try to be responsive to your needs and your requests. But sometimes, in trying to do this, we find that we’ve misinterpreted what you’ve asked for. Yes, it’s our job to ask questions in order to avoid such ambiguity. But we need the client to be clear and straightforward in voicing their requests.

Asking good questions. Seriously, we like it when you ask hard questions! Not nitpicky questions or gotcha questions but the kind that make us dig deeper and help to improve the quality of the end product. It doesn’t matter whether the question will require five minutes or five hours of additional analysis time, just that it helped all of us stretch our thinking and expand our insights.

How vendors price market research

One thing clients think they do not have a right to ask is how we go about pricing a given study. While it’s unfair to ask us to reveal our entire cost structure to you, there’s no particular mystery to it and it isn’t like we pull it out of thin air. Let me try to explain.

Of course, any professional services company has two primary methods available for how it prices its services. Law firms and consulting firms generally bill on a time-and-expenses basis. They’ll give you an estimate but in the end, they keep track of the number of hours spent, multiply them by each individual’s billing rate and add in any out-of-pocket expenses (sometimes with a markup). The other option is to quote a fixed price for a given project. Most purchasers of market research insist on the latter.

Some vendors use a hybrid approach, where most of the project is done on a fixed-price basis but with a few types of less predictable expenses billed on a pass-through basis. Examples of this might be travel expenses or respondent incentives. Often, these things are hard to predict in the proposal stage and so they are treated differently. It is also common for research firms to include a “plus or minus 10 percent” clause in their proposals, although many clients refuse to accept them, in which case we ask them to include a cushion in their budget in the unlikely event that we need to invoke this contingency.

How does the research firm go about determining a fixed-price quote? In the same way that they would if they had used the time-and-expenses method. We estimate all of our out-of-pocket expenses for things like rental of a market research facility, respondent recruiting, the use of an online panel, etc. Then we estimate the amount of professional time needed to execute the project, multiply those by our hourly or per diem rates and then quote the total. Billing rates are a function of salary and level within our organization (e.g., principals, project directors, analysts, etc.). If we execute the job more efficiently than projected, we win. If we execute it less efficiently, we lose. (Truth be told, the latter happens a lot more often than the former!)

There are a few other approaches to project pricing, although they are rare. Fifty years ago, it was standard to simply look at the estimated out-of-pocket expenses and multiply by three. The thought was one-third for out-of-pocket expenses, one-third for labor and one-third for profit. This always seemed pretty naïve to me, since the out-of-pocket expenses were often quite unrelated to the amount of professional time needed. For instance, it’s more expensive to do research in New York and Los Angeles that it is in Oklahoma City, even though it takes the same amount of time. And it costs a lot more to recruit doctors than it does people who eat breakfast cereal, even though it takes the same amount of time to analyze the quantitative data you might get from them.

A more sophisticated option is what people call value pricing – basing the price on what you think it will be worth to the client. While in theory this makes a lot of economic sense, it’s hard to do for market research projects. Estimating the value of information is a more esoteric exercise than estimating the value of a new piece of manufacturing equipment. Value pricing is one of those things that vendors like to think about in their fantasy world but reality usually dictates that you go with one of the more common methods described above.

Catharsis complete!

There, that feels better! Catharsis complete! In all seriousness, I hope that my insights here will prove helpful to both experienced and inexperienced purchasers of market research services alike. If followed, I’ll live longer and you’ll get a better end product.