Interpreting research data: It all depends on the context

By Ron Sellers | January 1, 1998

Reading time: 5 minutes

Abstract

Interpreting research data depends on the context. Often, the only context that exists is within the minds of company leaders and the expectations and desires may vary substantially between executives. This article presents a four-step process to help bring leaders together on expectations and desires while establishing a context for findings.

Research Topics:: Data Analysis | Qualitative Research | Quantitative Research | The Business of Research
Content Type: Research Methodology

Share Print

Editor’s note: Ron Sellers is president of Ellison Research, Phoenix.

Sixty percent of your customers just gave your company a top-box rating on a five-point scale of overall customer satisfaction.

Good or bad?

Well, if last year’s rating was 40 percent, this is probably cause for a celebration — unless the goal for this year was 80 percent. Similarly, if your competition is all at 80 percent, you may be in trouble. But if the competition hovers around 20 percent satisfaction, you look terrific by comparison.

It all depends on the context. It may be possible to compare your findings to competitive surveys or industry standards. In many cases, however, there is no readily available context. This is especially true when creating a baseline survey to track data over time. In future waves, you’ll be able to see changes in the data, but to start out, is your 60 percent top-box rating good or bad?

This is not just a problem with customer satisfaction surveys. Let’s say your company or client is relatively new to gathering information. A study discovers that 25 percent of your customers also purchase from the competition. Is this high or low? The same study shows that 50 percent of your customers have been doing business with your company for less than two years. Is this good or bad? This could suggest heavy recent growth (good), or it could point to high customer turnover (not good).

Often, the only context that exists is within the minds of the company leaders. Just as often, their expectations and desires will vary substantially from one key executive to the next. It isn’t uncommon to present findings in a meeting and have one person surprised at how high the satisfaction level is, while the next person is shocked at how low it is.

A very simple exercise can help to bring leaders together on this issue, while establishing context for the findings at the same time. This four-step process can be handled as follows:

First, agree on the most important findings from the study. Before the findings have been presented or made public, establish the key data points coming from the research. What are the five or 10 most important findings that will be the focus of the study? For a customer satisfaction study for a provider of value-priced consumer goods, a sample list might look like this:

1. Overall top-box satisfaction with the purchase.

2. Top-box satisfaction with the value received from the purchase.

3. Intent to repurchase.

4. Willingness to purchase from a competitor for a lower price.

5. Top-box satisfaction with the quality of the product itself.

These five data points, or others like them, may describe the most important elements of the study. This step is best accomplished with the researcher and the end users of the data, who will be setting the agenda for any changes that must be made as a result of the findings.

Second, have the team members each set realistic goals for what they hope the research will show. Each individual should write down what they would be satisfied with for each of the most important data points.
Third, have the team members each predict what they believe the research will show. This may or may not be the same number as appeared in step two. For instance, the marketing director may wish for overall customer satisfaction of 60 percent, but honestly believe it is closer to 40 percent. On the other hand, the product manager may be satisfied with a 60 percent rating, but believe that the real rating will come back as 90 percent satisfaction.

Even if these were the only three steps to the process, this exercise will often have substantial value. For one thing, managers frequently don’t know how to react to study findings because they didn’t know what to expect, and hadn’t really thought it through. In addition, it’s not uncommon for different leaders in a firm to have totally different expectations of what their customers will say. Much worse is that leaders may also have completely different views of what the company’s goals should be in these important areas. Steps one through three will alert them to these differences, and properly managed, this can mark the start of a process to crystallize company goals and expectations.

Finally, this will also help managers act on the research findings. Frequently, studies don’t get used to their full potential because different people come away with different ideas on what should be done with the information. If the end users are unified in their view of what is important, what their expectations are, and what their goals are as a company, the study is much more likely to receive action and attention when the data points to problems.

Fourth, compare goals and expectations with the actual study data. This will reveal gaps among expectations, goals, and reality. Let’s take the example given in step one, and fill in some sample figures from this exercise.

What do these figures show? First, the leaders already anticipate a higher-than-desired level of willingness to switch to a competitor due to price. They also believe their company will fall short of the overall satisfaction goal among customers. These two findings alone should open up some very useful dialogue among key managers in the company.

The findings should also be an eye-opener for these managers. First, their goals probably aren’t high enough in overall satisfaction, and their expectations of company performance certainly aren’t. There needs to be an exploration of why they felt the company is under-performing in these areas, and why goals are set so low.

Second, there is much more customer satisfaction with the quality of the product than they expected. Not only that, but concerns about customer defections due to price are far greater in reality than even in their fears. Finally, satisfaction with value was lower than they expected. The combination of these three findings suggests that their products may not be the value-pricing leaders the managers believe them to be. There is evidence here that although overall satisfaction is better than they believed, the company may be much more vulnerable than they thought to competitive price cuts.

Of course, the study will be more comprehensive than the five data points presented here. Assuming that careful examination of the complete data confirms these topline assumptions, what you now have is a leadership team in agreement about what the most important questions are, what the surprises were, and where action is needed. They have also begun discussing why they expected satisfaction to be so low.

In some cases, research may point toward necessary improvements in the product or service - in others, company leaders just need a more realistic view of what is going on in the marketplace. By having the end users of the data set forth their goals and expectations beforehand, you have a more useful set of findings as a result.