Using market research for business and public policy decisions in consumer lending

Monitor and measure

Editor’s note: Paul Lubin is senior vice president of Informa Research Services, Calabasas, Calif.

By regularly monitoring the experiences encountered by consumers applying for a loan, lenders, consumer groups and government can detect and resolve issues that inhibit the optimal allocation of credit. This in turn will help ensure the proper use of credit. For the lender, monitoring the customer experience can help prevent complaints and allegations that can negatively impact a lender’s reputation and sales.

Civil-rights groups, community activists, government regulators and enforcement agencies regularly use mystery shopping and post-application surveys to help detect violations of the law and acts against public policy. An example of this is the HUD Matched Pair Testing program, which assesses the treatment of minorities and non-minorities in the pre-application stage of the loan process. In addition, HUD provides funds to community groups to test for discrimination in lending. And there are the activist organizations, news organizations and class-action attorneys that use mystery shopping and other methods to test the sales practices of financial institutions.

Another example is the Financial Services Authority (FSA) in the United Kingdom, which regularly uses mystery shopping. The FSA is an independent body given the authority by the government to regulate the financial services industry. FSA has three strategic goals: promoting efficient, orderly and fair markets; helping retail consumers achieve a fair deal; and improving business capability and effectiveness. The FSA views mystery shopping as a necessary tool to help determine whether its goals are being met.

Critical window

First used in the retail sector to observe and improve retail conditions, mystery shopping now provides a critical window into the experience encountered by consumers applying for credit. It is a voluntary undertaking that can help minimize business and legal risk and ensure fair and safe credit practices. The procedure helps ensure consumer access to appropriate credit information, compliance with the law, adherence to business protocols and standards and thereby sound credit decisions by the consumer and lender.

The approach calls for the use of testers or mystery shoppers posing as potential or actual buyers. Unlike statistical procedures which require outcomes (loan approval, loan denial, pricing) and rely on abstract arguments and statistical principals, mystery shopping measures the experience encountered by the mystery shopper or tester. Mystery shopping can detect unfair or misleading sales practices or whether the experience differs based on race or ethnic origin of the mystery shopper.

Disparate treatment

Mystery shopping takes the form of matched-pair testing and monadic testing. With matched-pair testing, the objective is to determine the presence of disparate treatment or discrimination in the pre-application stage of the loan process. Matched-pair testing involves pairs of testers, e.g., minority and non-minority or male and female, posing as potential borrowers. Sometimes matched tests can comprise triads. Triad tests involve three testers: a control, the non-minority tester matched against two minorities (e.g., African-American and Hispanic).

The testers conduct their tests (shops) separately but each is provided with profiles or scenarios. The profiles of the testers are very similar or matched. In some cases the minority takes on a slightly better profile than the non-minority. The profiles include the purpose of the loan (most often first-time home buyer, though refinance and home equity/home improvement are also popular), loan amount, value and location of home, down payment and financial characteristics of the borrower. The only significant difference between the testers is that one tester is a minority and one is not, or one tester may be a male while the other is a female or one tester may be young while the other is old.

The focus of monadic testing is to detect patterns indicative of misleading sales practices and violations of the law including Section 5 of the FTC Act, which prohibits unfair or deceptive practices, as well as actions and patterns indicative of predatory lending. Monadic testing has become more prevalent with the increased number of product offerings, risk-based pricing and flexibility in underwriting in consumer lending.

Unlike matched-pair testing, monadic testing calls for the use of one tester and a specific test condition. Here the tester records his or her experience and the information and products discussed and the questions asked to determine whether the lender provided the necessary information for the consumer to make an appropriate credit decision.

Self-testing, whether performed on a matched basis for fair lending or a monadic basis for fair treatment, can be conducted either in-person or over the telephone. Most self-testing programs are conducted in-person. However telephone-based testing is frequently conducted when consumers apply over the telephone. Often concerns are expressed about testing for disparate treatment over the telephone. This author’s experiences, as well as research studies, have shown that in many cases race can be accurately determined over the telephone.

Quality of the assistance

Post-application testing is a customer feedback program which investigates the quality of the assistance and treatment provided to consumers (including minority and non-minority, male/female and younger/older loan applicants) after an application has been submitted. The procedure gained increasing acceptance as a valid method to test for discrimination and unfair treatment in consumer lending after a Federal Reserve study showed that race played a role in the mortgage decision and that differential treatment can occur at different stages of the mortgage process.

Conducting post-application surveys can help ensure fair and similar treatment of protected and non-protected classes in the various stages of the loan process. It can examine the coaching, advice, negotiation and competitive shopping and the degree to which it affects loan outcomes in the form of approval/denial, product and pricing. The approach can also help ensure fair treatment by monitoring the quality of assistance and the extent to which loan applicants are informed about and understand the loan products and terms of the loan.

The post-application survey is typically done by interviewing a sample of consumers (minority and non-minority) by telephone that has completed the application process. Applicants are interviewed to evaluate whether the quality of assistance provided by the lender affected the outcome (i.e., approval or denial). A sample of recent loan applicants who have received approval or denial should be interviewed. If you interview applicants prior to their formal notification, you will be unable to discern any correlation between how they were treated and whether their application was approved or denied. Moreover, they might misunderstand the purpose of your call and assume you are gathering more information with which to consider their application.

The post-application survey should be conducted by telephone. Other alternatives such as mail surveys jeopardize the reliability and timeliness of the information. Mail surveys are more subject to cooperation issues and the willingness of respondents to fill out and mail back the questionnaire. Some recipients may have no interest in returning the survey (for example, those whose applications were not approved) or they may not take the time.

Categorizing the sample by minority and non-minority group, product group or income group, as well as by approved and denied applicants, requires a large sample size. To make sure you have enough respondents, you may have to oversample - that is, interview more members of a group than the share they represent of your business.

Done right

To make sure the mystery shopping program is done right, the following guidelines should be adhered to:

• determine if assessment should be quantitative or qualitative;

• use comprehensive mystery shopper training;

• test realistic scenarios depicting real-life and high-risk situations;

• give thorough profiles to the mystery shoppers;

• design a questionnaire that gets you the information you need but is still easy to complete;

• carefully edit the questionnaires for completeness and accuracy;

• do not tell the mystery shoppers the objective of the study is to ensure compliance with the law and fair treatment of consumers (they should be told it is to measure service quality);

• limit mystery shopper detection by setting a schedule when the shoppers will conduct the tests;

• debrief the mystery shoppers;

• enter the data into an electronic database and generate crosstabulations of the data;

• analyze the data based on the objectives of the program and determine findings and implications;

• develop an action plan to ensure problem resolution and improvement.

Use these guidelines to maximize post-application telephone surveys:

• determine respondent definition and sample frame (e.g., interview consumers when experience is fresh in their minds);

• design a questionnaire that gets you the information you need but is still easy to complete;

• conduct follow-up interviews to explore problematic issues (either over the telephone or in-person);

• analyze the data based on the objectives of the program and determine findings and implications;

• develop an action plan to ensure problem resolution and improvement.

What can you measure with mystery shopping and market research?

• sales and service practices;

• treatment of minorities and non-minorities;

• the degree to which consumers feel welcome;

• needs discovery, ability to repay and suitability of products offered;

• advice and coaching provided;

• access to service and information;

• courtesy and comfort; and

• office and employee performance.

Powerful tools

The above mystery shopping and market research methods are powerful business and public policy tools. All businesses and government agencies involved in consumer credit owe it to consumers - and to themselves - to use these and other techniques to uncover prejudice and unfair sales practices. The costs to use them are not great. The techniques are time-tested and akin to the customer surveys many companies already perform. The costs of not using them, however, can be prohibitive.