Mystery shopping, as initially used by industries, was unsophisticated, lacked reliability and, in most cases, was statistically unsound. The goal of early programs was to evaluate retail conditions - that is, for example, how products were sold and promoted or how prominently a product was displayed and where.
These early mystery shopping programs were deemed "observational" because the program objective was to have the shopper observe and record what he or she saw. In translating the observational approach to the retail scenario, this approach has advantages but also some drawbacks. By making observations only, it is difficult to evaluate the interaction of a company’s staff and customers. In other words, some of the mystery is taken out of mystery shopping.
Mystery shopping became much more useful as a self-assessment tool when mystery shoppers - or testers posing as customers based their evaluations not only on observations but also upon actual retail transactions or inquiries — for example, asking about or purchasing a product from a sales clerk at a department store, opening an account at a bank, or cashing a check at a teller station.
Although mystery shopping was often initially thought of as a very subjective and qualitative research technique, by increasing sample sizes (number of shops) and adding an objective (checklist-yes/no) questioning format, the findings became more reliable. Therefore companies began relying on mystery shopping more and more. Indeed, the information and sales age of the 1990s has brought with it an increasing reliance on mystery shopping to monitor sales and service performance during customer-employee meetings.
Most retail companies and financial institutions got their feet wet in market research via mystery shopping programs to evaluate the sales process, cross-selling, product knowledge and customer courtesy. These programs were usually conducted annually or at least every two years. They served as benchmarks or baselines and as follow-up programs to evaluate change.
But by the 1970s, companies realized that mystery shopping done infrequently did not serve a motivational purpose; it was nothing more than an expensive monitoring tool. By this time, the retail industry had begun using shoppers on an ongoing basis. The result was that store managers and employees, aware that mystery shoppers might visit, were more careful about how they treated customers and how they displayed and sold their products.
Many service providers such as banks, hotels, etc., took a page from their book and also begin using the mystery shopping technique on a more or less continuous basis. By having mystery shoppers visit stores and/or branches quarterly, monthly, and even weekly, the technique not only evaluates sales professionalism and service quality but also serves as a motivational tool.
Matched pair testing
In more recent years, banks, mortgage and finance companies have come to recognize the importance of monitoring compliance with the Fair Housing Act and Equal Credit Opportunity Act for evaluating equal treatment, disparate treatment and overt discrimination in lending.
The banking industry uses a technique called matched pair testing to ferret out discrimination. Borrowed from civil rights advocates and fair housing rights groups, the technique has become so popular that it is now recommended by the Department of Justice as well as other banking regulatory agencies. The government has gone so far as to protect or privilege the information collected via matched pair testing from discovery in litigation for banks.
Other service sectors such as the brokerage and auto industries are also using mystery shopping to ensure that their sales practices are fair, equal and not misleading.
Matched pair testing is a classic form of self-evaluation. It is similar to mystery shopping programs that are designed to evaluate sales practices, sales professionalism and skills and service quality/courtesies, etc. It involves sending pairs of testers (shoppers) posing as potential borrowers (if it is a bank) potential investors (if it is a brokerage firm) or potential retail customers, patients or hotel guests, into the field. They conduct their tests (shops) separately, but each is provided with similar profiles or scenarios. The only significant difference between them is that one tester may be a minority and one is not, or one tester may be a male while the other is a female or one tester may be young while the other is old.
Although a well-designed matched pair testing program can provide information about service quality and sales professionalism, the primary objective is to evaluate and compare the treatment each tester receives and if the treatment is equal.
Risk is high
Most companies don’t recognize discrimination’s effect on sales and marketing. The risk associated with discriminatory practices is high and can take different forms — financial liability, legal violations and lost business. Bias charges can be made regarding equal employment opportunity, equal credit opportunity and equal access to goods and services.
Shoney’s restaurants agreed to pay $105 million to black employees and job applicants who claimed they were discriminated against. Another restaurant chain, Denny’s, agreed to pay $54 million to settle claims that it denied service to black customers.
More recently, Texaco agreed to pay $176 million to African-American employees because of charges the company denied promotions on the basis of race. Texaco also agreed to increase employment of minorities and women. These companies faced negative and damaging press that drained their goodwill in the community and set back sales and marketing.
Uncover bias
In addition to mystery shopping, other forms of market research (e.g., fair treatment surveys and statistical modeling) can measure: whether marketing discourages minority business; minority and non-minority access to a company’s goods and services; service bias towards whites or men; access to products made available to different consumer groups; and the role race plays in various business decisions.
In most cases, discrimination can be classified as either overt or subtle. Overt discrimination denies minorities the same opportunities offered to whites. Subtle actions are more difficult to measure; they encompass assistance issues and the level and type of accommodation but can be ferreted out via matched pair testing.
Matched pair testing represents a powerful approach for detecting discrimination in the pre-application or purchase phase of the sales process. Testers or mystery shoppers matched in every possible way, with the exception of the treatment tested (e.g., race or ethnic status), measure the sales process in terms of access to goods and services encouragement, information and types of products and terms offered, courtesy and time spent. The methodology tells you whether discrimination is occurring and whether or not your sales practices are limiting sales.
Correct problems
The nature of your business and the way your company markets its products determine how you should test for discrimination. Matched pair testing helps companies detect and correct problems, practices or policies that may represent a risk, just as product testing is self-testing because it helps companies limit consumer rejection and identify potential product defects that can lead to lawsuits.
There are plenty of reasons to conduct matched pair testing and if you do, you can do so with confidence. The fees aren’t great considering the risk associated with discrimination and the technique is time-tested and widely accepted by business and government agencies.