Editor's note: Dan Prince is founder and president of Prince Market Research, Nashville, Tenn., a specialist in mystery shopping and customer satisfaction surveying.

A mystery shopping program will sooner or later grind to a halt if managers and employees claim the program is not objective. Conversely, if the shops are perceived as being fair and unbiased, then you gain employee buy-in and a much greater chance of instilling desirable behavior by your frontline staff.
Starting with a single banking client, we implemented our first mystery shopping program seven years ago. Since then, we have served additional clients in financial services, retail, and health care. While a portion of our mystery shops are performed in-person in several different states, a growing number are now being done over the telephone.

Out of the experience of serving these companies - each of whom demands accurate, objective information from us - we have developed several practices and guidelines which may be useful to you.
To ensure an objective mystery shop - and a credible mystery shopping program - our experience suggests that you must:

  • design a scoring sheet that promotes objectivity;
  • screen potential mystery shoppers carefully;
  • try them out (on a competitor);
  • review each completed scoring sheet in detail; and
  • be prepared to replace suspect shops with a re-shop.

1. Design a scoring sheet that promotes objectivity.
While this may not seem to be the obvious first step, it is. In fact, we have found that is the most critical step. If you don't design a scoring sheet that allows for objective grading of an employee's performance, then you leave yourself open to wide interpretation of what constitutes acceptable behavior by employees.
Mystery shopping is built on the assumption that your client (or your company's senior management) knows and can articulate a set of behaviors that, when delivered with sincerity, constitute a positive, productive interaction with a customer. It's what sometimes called a "service script."

At the beginning of a new shopping program, we work with our clients to write out the statements of desirable behaviors which constitute the company's service standards. It's very important to be sure that these written standards have buy-in from relevant senior managers - before further developing a shopping program.

For example, in a retail banking environment, management may decide that a customer service representative needs to do each of the following things at the beginning of an interaction with a customer:

- stand when a customer approaches his/her desk;

- make eye contact and smile at the customer;

- introduce him/herself;

- ask the customer's name;

- shake the customer's hand;

- offer the customer a seat.

While managers may agree easily on most of these behaviors as desirable, there could be a debate on a particular item. For example, we have had bankers that argued rather hotly whether it was necessary to shake the customer's hand in every case. In one instance, bank employees felt that older female customers were not used to shaking hands. In another case, bank employees said they didn't like shaking hands with people who were "dirty" (like construction workers who come into the bank right from the job site). Management needs to consider these kinds of issues, then make a decision.

Generally, if management leaves it up to an employee about how to act in a certain situation, then management is foregoing the opportunity to make that particular behavior a service standard. Accordingly, it should not be included on the scoring sheet used by a mystery shopper.

You'll note that each of the five statements on the list above is a specific and observable behavior. This is much different than asking the mystery shopper to tell you if the employee was courteous at the beginning of the conversation. We say, "To be measured and managed, it must be observable."

By carefully defining the behaviors that constitute a courteous interaction, we get an objective gauge of whether employees are being courteous rather than an subjective opinion, which could vary widely from one shopper to another. The same approach applies to other behaviors as well, such as consultative selling skills which involve asking questions to determine the customer's need, presenting alternatives for the customer's consideration, recommending one, asking for the business, and so on.

Once the specific desirable behaviors are listed and described, then a scoring sheet can be created. On the scoring sheets we develop, we do our best to list each of the behaviors in the order they will likely happen during the interaction between the shopper and the employee. In the field, this will make it easier for the shopper to recall the interaction as it unfolds, and therefore, score it more accurately.

For example, in serving a customer in a bookstore, management has decided that a floor employee should (1) ask questions to determine the customer's area of interest, (2) take the customer to the appropriate section of the store, (3) show the customer several alternative titles, (4) help the customer decide which one(s) to buy, and (5) ask if there is something else they can help the customer with.

Thus, to lay the foundation for an objective and credible shopping program, start with the specification of the service standards and the development of a detailed, behavior-oriented scoring sheet.

Hint: Have staff people take your draft mystery shop scoring form into the field and try it out. Generally, we conduct a trial with at least five shops so we get a good feel for how well the scoring sheet stands up under various situations. The task here is to be sure that the statements on the form are clear and not subject to misinterpretation.

2. Screen potential mystery shoppers carefully.
When I meet someone and mention our mystery shopping activities, frequently the first question asked of me is "How do you find people to be your mystery shoppers?" (The second often is, "Can I be one? It sounds like fun!")

Finding a mystery shopper goes well beyond identifying someone who has an interest in playing the part of a customer. We have clients who carefully review every scoring sheet we turn in to them - and some make shopping results part of their managers' performance reviews.

Consequently, we must take steps to ensure not only the overall integrity of our results and reports, but the integrity of every shop we accept for data entry and tabulation. This means finding people who can do this job faithfully, accurately and objectively. If you haven't tried it, you will likely find that this is harder than it sounds! We run newspaper ads and also rely on temporary employment agencies to provide a pool of interested, potentially qualified mystery shoppers. Then we meet with the candidates personally to assess how well they fit the task.

The key qualities we are looking for include:

  • No obvious bias for or against company to be shopped.

    Hint: Before telling a candidate what business will be shopped, ask a series of screener questions in order to disqualify people who have a grudge against a specific business or type of business. Also ask questions to disqualify people who have a direct family member working in the business to be shopped.
  • Ability to "role play" the desired customer role - This includes responding to questions that might naturally come up (e.g., asked of a new customer opening a bank account: "So you work at ZZ Technologies. Do you know Sandy Lehner?").

    Hint: During recruiting, role play situations that have (or could) happen in the field in order to see how different candidates react. * Ability to be neutral throughout the transaction
  • In their interactions with employees, whether in-person or by phone, shoppers need to be neither too friendly nor too distant. For example, we once encountered a person whom I'll call Sally Sunshine. She passed our initial screening process and was sent out to perform her first set of real shops. When she returned, it was clear from both her written comments and her verbal remarks to us that she was having a tough time being objective. She consistently gave retail store employees scores that were too high based on the behaviors actually encountered, explaining that this was because certain employees "seemed so nice."

    While we often add a final scoring category regarding the overall courtesy and helpfulness that is intentionally intended to get at the feeling a customer/shopper gets from the employee, this is typically the only place that being extremely nice to a customer gets scored.

    Alas, Sally didn't make the grade.

    Hint: Conduct a role play in a group or one-on-one training situation. Give the candidate a scoring sheet and ask him/her to score an interaction that you stage, and then see if they score it correctly and what they have to say about the interaction they have just observed.
  •  Ability to accurately recall the details of the experience - This is critical in order to complete the scoring sheet after leaving premises. Our shoppers must write a short commentary of key points in the service encounter as well as check and score the behaviors we have agreed to look for.

    Hint: Use the same exercise as the one described above to judge this. Plus, you can send them into the field on a trial run...which brings us to our next point.

3. Try them out - before turning over a set of shops to be done.
Before you assume that you've got a shopper who can be accurate, thorough and objective, send them out for a few trial shops. If possible, go with them and observe the interaction they have with an employee being shopped. Then have the shopper fill out a scoring sheet and see if they score it like you would have.
This will tell you very quickly whether this person is likely to be an effective mystery shopper. It also affords an opportunity for the potential shopper to bail out if they don't like the work, too, before you've placed your trust in them to go out and complete 15-25 shops.

We send a "shopper in training" to a competitor's location, not to our client's place of business. (The same idea works even better for telephone mystery shops.) This way, if the shopper doesn't do a good job, we haven't compromised the quality of our shopping program.

An interesting side note: Because our firm conducts a large volume of customer satisfaction research, we've found that some individuals can do a good job of conducting telephone interviews and also performing telephone mystery shops. Ironically, for us, this has been the exception rather than the rule, however. Most people turn out to be good at (and like to do) one or the other, not both!

4. Review each completed scoring sheet in detail before approving it for data entry.
Since staff time is involved in this step, this may be the one that gets overlooked or shortchanged. Soon after the scoring sheets arrive at our office, we check each sheet, in detail, before sending it along for data entry and tabulation. Here are some of the checks and balances we use:

Make sure that the checked items are then scored correctly. Compare what has been written in the comments section to the scoring.

Call the shopper to discuss/resolve any apparent discrepancies. If we find discrepancies or omissions, we contact the shopper immediately to discuss the shop and its scoring. If the shopper can recall the details sufficiently, then we make corrections or changes to the scoring sheet. If not, then we feel that we have no choice but to follow step five.

5. Be prepared to throw out any suspect shops - and re-shop these using another mystery shopper (or in a deadline situation, a staff member). This is your final insurance. Being willing to throw out a "suspect" shop and re-shop the same location and/or employee ensures the credibility of the results you provide to your client. Being willing to do this, and do it at your own expense, reinforces the belief that the work done by your team reflects objectively what is actually going on out on "the front line."

Following these steps goes a long way toward providing objective and credible shopping results. And with these results, management is in a position to reward desired behavior, prioritize training and coaching needs, and over time boost the level of sales and service performance by frontline employees.