Measure it and communicate it
Editor’s note: Paul C. Lubin is senior vice president of Informa Research Services, Calabasas, Calif.
Research has demonstrated a link between customer experience and customer satisfaction, purchase decisions and, ultimately, commitment to the brand in terms of customer advocacy and longevity. Highly satisfied customers have a stronger relationship with the company, they generally buy more or own more company products, are more likely to recommend and are more loyal with greater tenure. All this contributes to and increases brand equity and the lifetime customer value.
The impact of an inconsistent and sub-par customer experience leads to customer dissatisfaction, customer attrition, diminished product purchases and ownership and increased expenses per customer, all of which erodes the lifetime value of the brand.
For customer experience marketing research to help win customers, and play an important role in your company, the information must be accurate, actionable, timely, easy to understand and above reproach. Here’s how to ensure that your customer experience research project meets all of these criteria.
Popular methods of measuring the customer experience
Mystery shopping measures the experience encountered by the mystery shopper or tester in a retail setting. The approach provides insight into the way products are sold and transactions executed. Mystery shopping does not rely on the recollection of the customer, it records actual behavior, monitors the information employees provide to customers, collects product literature and documents retail product purchases and transactions.
A mystery shopping program helps the company ensure consistent, sound and superior sales and service practices by measuring behavior at the point of purchase and by providing feedback to employees concerning their performance and adherence to company standards and protocols. On a strategic and tactical level mystery shopping provides an overall assessment of the company’s strengths, weaknesses, opportunities and threats and thereby helps ensure a customer experience which meets client needs and is superior to the competition.
Customer satisfaction surveys measure the degree to which customers are satisfied, whether needs are met and why and where the company can improve.
Baseline and diagnostic customer satisfaction programs provide an in-depth assessment of client satisfaction as well as customer loyalty and advocacy. The surveys determine customer needs and how customers form their attitudes toward the company. The surveys pinpoint which aspects of service and products impact customer satisfaction and loyalty and whether to recommend the company. Baseline or diagnostic customer satisfaction assessments also compare a company’s strengths and weaknesses versus the competition and therefore measure the value offered by the company relative to other companies. Information obtained from these studies is used for strategic and tactical purposes. The research identifies gaps in the company’s product and service offering and helps guide decisions where to improve and evaluates the company’s progress over time.
Customer transaction-based surveys have gained popularity over the last 10 years. The programs interview customers shortly after a transaction and assess overall satisfaction and satisfaction on a few critical aspects of product and service and provide for follow-up to resolve problems. These programs help a company communicate to employees and customers that it cares about customer service.
The results from mystery shopping, baseline and customer transaction surveys are often considered part of the key performance metrics of the company and are integrated into employee recognition programs in order to motivate and recognize performance.
Customer attrition surveys represent an in-depth review into the reasons for lost business and customer dissatisfaction. The surveys help the company identify a direct link between service- and product-related issues and lost revenues. Customers who closed accounts or substantially reduced business are interviewed shortly after the decision to determine the reasons.
Too broad
Defining and setting the objectives are the first steps in conducting a successful customer experience marketing research project. Many times company personnel develop objectives that are too broad. An example is management launching research to measure customer satisfaction. The objective does not set guidelines for how the information will be used, thus making it difficult to determine the most appropriate data collection method. Will it be used to assess the company’s core value positioning or is it a program to help improve at the store or branch level?
Vague objectives can lead the researcher to select a research methodology that does not meet management’s underlying goals. For example, an objective to determine satisfaction with the company’s core value positioning is very much different than a program meant to ensure satisfaction at the store level. Each calls for a specific type of interview, sampling frame and method, and number of interviews. Even the manner by which the data is collected and reported needs to be assessed.
The research plan needs to specify how the information will be used and the standard by which the information will be judged. This allows an evaluation of different data collection methods as well as the proper number of interviews, correct respondent definition and questionnaire content. A meeting between the market researcher or market research department and the intended user of the information is usually a big help. During this meeting the parties should define the issues and questions that need to be answered, standards upon which action will be taken and when the information is needed.
Vitally important
Selecting the right method for data collection is vitally important to meeting the research objective. Without a proper method for gathering the data, the findings produced by the research project will be flawed and inappropriate decisions can be derived from the data. The researcher should review the objectives and the information that needs to be collected as well as any critical decision criteria that will be used in determining the most appropriate data collection method.
With mystery shopping, unlike an interview with a customer or consumer where the interviewer records information obtained from a respondent, the shopper or interviewer is the respondent since he or she poses as a customer or consumer and records the information provided by the employee. If conducted properly, mystery shopping provides information as to what really happens when the company representative meets or speaks with customers and non-customers. Mystery shopping also provides one of the few ways to measure the sales process and determine why prospects reject. Even if there were an accurate prospect list, the survey would be limited due to lack of cooperation and the prospects’ ability to remember what happened.
Telephone surveys still offer representative samples and the ability to quantify information such as customer satisfaction, expectations and attitudes and probe into the reason for a consumer’s beliefs or actions. The ability to probe and clarify consumer responses during the telephone interview is a key strength. Surveys can be conducted through random-digit dialing of telephone-owning households in a defined area or via customer lists with telephone numbers. Cooperation rates have declined recently in step with consumer concern for privacy and issues related to telemarketing. Using a customer list and providing the company’s name at the beginning of the survey and keeping the interview short helps ensure cooperation.
The use of Web-based surveys continues to grow. Many panels of consumers exist to conduct Internet-based research. The Internet panels are particularly helpful for surveys targeted at hard-to-reach consumers, e.g., doctors or other busy professionals. They fall short when the company requires information on branch-based service since many companies do not have accurate and sufficient numbers of customer e-mail addresses.
Mail surveys are cost-effective, provide a framework for thoughtful input and provide a sense of comfort for respondents since they know the survey is from the company and that they can provide the information in private. Lower cooperation rates are typical versus other data collection methods. A key point to consider is whether the consumers who respond to the mail survey are likely to have the same opinions and attitudes as those who do not respond. If the answer is yes, then the data gathered from the respondents returning questionnaires should be representative of the population. Another key point to consider is timing. Mail surveys typically take longer owing to the time it takes to mail questionnaires, mail reminders to complete and return questionnaires, and then receive the completed questionnaires.
Personal or exit surveys are either conducted in a branch or store, when a customer leaves a store, or in a shopping mall setting. The branch-based interviews are particularly helpful for targeting customers of a branch and gathering information about the store experience when it is fresh in a customer’s mind. Shopping mall surveys are helpful for intercepting consumers to participate in a survey requiring some type of visual stimulus. The branch and shopping mall surveys, however, only represent the consumers visiting the branch or mall that day and, hence, the user of the data needs to assume that consumers who visited the branch or shopping mall that day will provide the same information as those that did not.
Focus groups are particularly helpful for understanding customers’ views and decision-making and are often a prelude to a quantitative market research project. The information, however, is qualitative in nature since it is based on a discussion with small groups of consumers. The opinions voiced during the discussion can often be colored by one or two dominant participants and should not be viewed as representing the company’s customer base. Therefore decisions should not be based solely on the information produced by focus groups.
Depends on the purpose
The number of interviews completed depends on the purpose of the research and the decisions that will be made using the information. To ensure that the information is actionable, the researcher must complete a certain minimum number of interviews to provide management with a level of confidence for making decisions. Before launching the research, the researcher should set action standards for the research and the information it provides.
Many times the number of interviews conducted is based on cost. Statistical reliability and setting action standards to make decisions take a backseat to cost and the time the information is needed. This can seriously detract from the credibility of the research. Verifying the number of interviews will meet the action standards and how the data will be used is vitally important.
The number of mystery shops must be sufficient to meet the objective. If the objective is to ensure consistent and superior sales and service performance then a sufficient number of evaluations of the sales and service experience should be completed. An action standard should be established and performance judged based on meeting this action standard. An example would be setting standards or protocols of sales and service, communicating those protocols to employees and then judging company-wide performance relative to those standards. Performance gaps can then be determined and weaknesses addressed at the company level.
The more evaluations the better. While 100 evaluations may be sufficient at the company level it may not be sufficient to evaluate performance at division, district and store levels. Setting quotas or number of evaluations by division, district or store will help with reliability. At the store level, recognition should be carried out with a minimum of 50 evaluations. Coaching and using the results in performance reviews can be conducted with fewer evaluations but the more evaluations the better.
For customer surveys, the number of interviews completed depends on the purpose of the research and the decisions that will be made using the information. To ensure that the information is actionable, the researcher must complete a minimum number of interviews to provide management with a level of confidence for making decisions. Before launching the research, the researcher should set action standards for the research and the information it provides.
One thousand interviews will provide for greater reliability and the ability to review results by subgroup than 100 interviews. For example overall customer satisfaction might be viewed as weaker among affluent customers than middle-income customers if there is a difference of 9 percentage points between the two groups assuming 250 interviews each at the 95 percent confidence level.
If the customer experience program is meant to recognize performance at the store level the company should carefully consider the number of interviews completed by store. Interviews should be conducted monthly and recognition provided at no less than 30 interviews per branch with the objective of completing 100 interviews or more per store over the course of the year.
Little relevance
Ensuring the questionnaire is easy to understand and administer and free from bias is vitally important to collecting action-oriented information. Marketing research surveys are often too long and contain questions with little relevance to the research objective.
Key points to remember are to keep the questionnaire short, concise and easy to understand and administer. It is important to ask the right questions in an order free from bias. The answers customers provide in the beginning of a survey influence the answers they provide later in the survey. The order in which questions are asked can bias the results and provide misleading information.
Asking a customer how satisfied he or she is with the institution at the end of survey may result in a completely different response than asking overall satisfaction at the beginning. The satisfaction measure taken at the end of a survey will be biased, as the customer will have been introduced to a variety of topics which may or may not be important to them. Asking about their level of satisfaction at the beginning of the questionnaire limits bias as the customer will think only of the topics important to him or her when answering the question.
While mystery shop questionnaires are less prone to redundancy or covering the same topic with different questions, they too need to be easy to understand, clear and concise. A big help is to order the questions in the same order as the sales and service process. This helps the shopper complete the questionnaire accurately and remember the details of the shop. For example, do not ask if the representative introduced himself or asked for the shopper’s name at the end of the survey when the representative is most likely to do this at the beginning of the meeting. Unlike a consumer survey, where it’s important to obtain overall impressions up front, the more subjective or satisfaction questions are best asked at the end of the questionnaire in a mystery shop exercise. This helps limit bias introduced by what the shopper feels is important. The shopper completes the satisfaction ratings after completing his or her observations and therefore should be judging the representative based on these observations.
Make or break
The reporting and analytical approach can make or break a customer experience research project. The approach for reporting and analyzing the data must meet the objectives of the research. It must describe the key performance metrics and decision criteria and point out patterns in the data important to the key decision makers. In a research program designed to motivate branch service relative to goals, it should quantify customer satisfaction and report on the level of service quickly and in an easy-to-understand format. It should tell the staff what they do well and the areas where they can improve. Dashboards showing the key performance metrics work well. These might include the company overall score, scores on key performance variables, the branch or store score (for the store manager) and stores ranking in the top 5 percent.
An analytical scheme that works well in mystery shop programs is quantifying and then reporting performance and the consistency of performance across locations and representatives and comparing the scores against a benchmark or industry standard. The more inconsistent the service the more risk to the brand and revenues associated with customer service. Another approach calls for mapping the company’s performance and consistency in the service experience and comparing to peers or a normative data set.
Many times, customer experience information will be used for inappropriate purposes. A common misuse is when a company uses a program designed to motivate employee service by measuring satisfaction with recent transactions to determine overall customer satisfaction and loyalty. Customer satisfaction with the company and its value proposition can be very different than satisfaction with recent transactions. Inappropriate applications of this sort can lead management to question the results and the usefulness of the data collected.
Wins customers
By following the above steps, researchers can help ensure that their customer experience marketing research wins customers and that management views the research as an action-oriented decision-making tool that is vital to the success of the company.