Ted W. Allen directs market research for Bell Atlantic Directory Services, Bethesda, MD. He received his M.B.A. and B.A. (psychology) from the University of North Carolina at Chapel Hill . His Ph.D. and master of science degrees are from the University of Michigan (Ann Arbor ) in cognitive psychology. Prior to joining Bell Atlantic , Allen served as a professor in the UCLA psychology department. He has also served as a research scientist at the American Institutes for Research in Washington , D.C. He has also worked as an independent research consultant assisting in the development and marketing of information products.
“For us, good market research is not a luxury, it's a requirement," says Robert Graham, president of Bell Atlantic Directory Services.
"Keeping Bell Atlantic's Yellow Pages the most-used directory in a highly competitive environment is a tremendous marketing challenge," Graham continues. "Especially when you serve every household with a telephone, and markets vary from the very rural—with a 40 page directory, to the very urban—with a 3,000 page book. We must have good market research."
Bell Atlantic Directory publishes some 300 Yellow Pages directories throughout its region (Delaware , Maryland , New Jersey , Pennsylvania , Virginia , West Virginia , and the District of Columbia ). In 1985 Bell Atlantic entered the specialty directory market and now publishes such highly regarded products as the Washington , D.C. Visitor's Guide and the Security Industry Buying Guide (the latter in cooperation with the American Society of Industrial Security).
Competition for the directory business in the Bell Atlantic region has nearly doubled in the last four years. In 1984 Directory competed against 24 publishers who were responsible for some 200 directories. By 1988 there were over 40 competing publishers responsible for some 350 directories. In practically every major urban market, Directory faces substantial competition, with the threat of more competition ever-present. Competitors vary from small, specialized operations to information services and telecommunications firms in the Fortune 500.
Market research and competitive analysis have become the foundation of Bell Atlantic Directory's efforts to meet the challenge posed by the entry of competitors into the directory business. According to Graham, "People use our Yellow Pages an average of 10 times more than those of our competitors. And they consistently rate our specialty directories as excellent. Good market research has played an important role in keeping these numbers high."
Information management
The goal in market research has been to build a small, efficient group that could provide effective research and decision support for all directory products. Key to this unit's success has been the emphasis on the quality and flow of information in the decision process. The basic premise is that good marketing decisions will be made when accurate and reliable information is provided to the persons best able to use that information. This approach has been an important part of Bell Atlantic Directory's market research from the outset. When the market research group was first established, Graham and his strategists considered three conditions to be necessary for the group to be both efficient and effective.
First, market research information had to be managed with the same philosophy that the company applied to any other information that it collects. Just as no two locations would be allowed to collect accounting information differently without a valid business reason, market research, too, would have to be centrally coordinated for Directory's products. Further, there would be a strong and clear policy for the dissemination of that information. Market information had to get to the managers who needed it, but it also had to be safeguarded from those who might misuse it.
Second, consistent, top-quality research would be difficult to achieve if the research managers merely gathered research requests and passed them on to vendors. Therefore, the research managers had to actively direct the research process, and that required a high level of technical competence.
Further, the technical expertise had to be broad, since each manager would be responsible for all phases of their research projects. The manager would be the client's single point of contact throughout the project, being responsible for the research design, questionnaire development, data analysis, and research interpretation. This concentration of roles and efforts results in more powerful research and more efficient use of resources.
The third requirement was that the market research managers have a sound understanding of the directory business. The market research unit provides research and decision support for all business units, including production, delivery, sales, strategic marketing, pricing, and finance. To produce effective research, the research manager has to know enough about each of those aspects of the directory business to translate a business need into the requisite research design and data presentation format.
"In an information industry such as the directory business, the way that you manage market information is especially critical,” explains Elizabeth Peterson, director of the Strategic Marketing Group in Directory. "You have got to collect the right information and then present it to your executives, project managers, and customers in a format that meets their business needs. Our approach gives us that capability and makes market research central to Directory's decision process." The remainder of this article focuses on how Bell Atlantic Directory implements this approach in data support functions.
Data support
Bob Graham reports, "We have met with a wide variety of research suppliers and syndicated research organizations. We are consistently told that our standards for reliability and accuracy are among the highest in the industry. Frankly, we are very proud of that and intend to maintain those high standards."
Proper data support ensures that the market research presents an accurate picture of the market place. These functions are the foundation of Bell Atlantic Directory's market research information system.
At a basic level, data support means a concern about sample size, sample generation techniques, data cleansing, and all the other factors involved in providing reliable, high-precision data. For most studies, the data support process consists of at least two major functions.
1. Sample development. These are the procedures through which decisions are made about how the sample will be drawn, who will be included, and how many respondents will be interviewed. At Bell Atlantic Directory these decisions are driven by how the data will be used once the study is complete.
For example, if the data will appear in promotional material, a larger sample size might be used to increase the precision of the numbers and to generate greater confidence in the data. In contrast, for internal modeling, the sample size depends upon the overall precision of the model. Thus, if the other data in the model had wide margins of error, there would typically be little benefit in using a very large sample to collect relatively few of the model's data points.
Just as important as the size of the sample is the pool from which the sample is drawn. There are two major concerns: (1) insuring that the sample is truly representative of the people to which the product is targeted and (2) clearly identifying the respondent's impact in the market (e.g., determining whether the respondent is a user, purchaser, decision-maker, signatory or influencing agent). The latter identification is especially critical in strategic studies where the data must indicate both the direct and indirect effects of the respondents' attitudes.
2. Data maintenance. These are the procedures involved in assuring that the data accurately reflect the responses and behavior of the persons interviewed. Typically, these operations involve checks for accuracy and consistency in the coded data.
Since Directory's market research is usually conducted by independent research vendors, the vendor plays a critical role in data maintenance. Data maintenance begins with vendor selection.
Interviews, detailed proposals, and onsite visits play an important role in Directory's assessment of the quality and care of the vendor's operations. Recently, Directory has included a vendor capability and operations statement in its assessment tools. This statement is a formal, written, and detailed description of the vendor's operating procedures and capabilities. All vendors interested in doing business with Bell Atlantic Directory are required to complete this statement.
The vendor capability and operations statement is then included as part of all contracts, with the vendor agreeing that operations will be carried out as outlined in the operations statement. Thus, there is some assurance that the quality of the vendor's operations will be maintained as they were when the purchase decision was made.
Data verification is the second major area of data maintenance. Typically, Directory receives a copy of all raw data which is used for archives, further analysis and data verification. In data verification, the vendor's analyses are duplicated using the raw data to test the accuracy of vendor-supplied tables and analyses. Any discrepancies must be resolved before the data are used.
As Charlene Gordon, one of Directory's research managers, reports, "Data verification simply has to be done, even with the best vendor. In our annual usage study, which we use to show advertisers how much the consumers are using our directories, we spend at least a couple of months checking the data and tables for the more than 6000 respondents. It is time-consuming, but it is the only way we can be assured of the accuracy of our data."
An additional, often difficult area of data maintenance arises from the analyses project managers run on the data once they receive it. The easy availability of cheap electronic spreadsheets and calculators has tempted some project managers into doing their own analyses of market research data. Unfortunately, some of these devices and the implemented equations have questionable numerical accuracy.
Where clients indicate that further analysis or modeling is to be applied to the data, the market research process is extended to provide assistance in those analyses. Further, certain data such as that included in promotional material must be approved by the market research group before the material can be distributed.
Validity and reliability assessment
Directory market research is usually directed toward measuring attitudes and behaviors, such as directory usage, through some interview mechanism. Unfortunately, respondents do not always report their true behavior and attitudes. The third major function in data maintenance is concerned with insuring that the data reasonably represent actual behaviors.
In measuring consumer usage of two competitive directory products, for example, one could ask:
- "How many times did you use X directory in the past month?" versus "How many times did you use Y directory in the past month?"
- "How many times did you use X directory in the past week?" versus "How many times did you use Y directory in the past week?"
- "Which directory, X or Y, do you use most often?"
- "Which directory did you use last?"
- "Which directory do you prefer, X or Y?"
Though these measures are frequently quoted, the questions are not equally valid in measuring usage. Measure 1 for a particular directory almost never equals four times Measure 2 for that directory. Measure 3 can show results different from the comparisons in Measures 1 and 2. And for a number of reasons, Measures 4 and 5 appear to have the weakest validity for total comparative usage.
The other side of a "good" measurement tool is that it be reliable and provide similar results when repeated under the same conditions. As might be expected, the five questions described above also differ in reliability. Measure 4, for instance, might show a change when other measures show no change.
Database design and maintenance
For Bell Atlantic Directory, the key to valid and reliable market research data lies in intelligent design and maintenance of a good market research database.
For some studies, conceptual models are developed at the design stage. For example, a model might describe each behavior and motivation in the usage process and then indicate measures that would capture the behavior. There is then a basis for believing that the interview really is providing data about actual behavior.
Also, Directory's studies typically contain multiple measures of the critical variables. Rather than ask one question about usage, a number of questions are asked. Correlations with other measures and other studies can then be used to help pinpoint the strong and weak measures. Even something as simple as checking to see if the numbers fall into the same range as in related studies can help to pinpoint problems in the data and change the weight attached to each measure. Having a good database from past studies is critical for this. The data maintenance procedures are crucial in providing top quality data to Directory's managers and customers.
Success requires technical competence and excellent vendor/client communication.
Summary
Market research has played a key role in Bell Atlantic Directory's competitive effort. Directory's information management approach to market research has placed a strong emphasis on providing accurate and reliable data to the appropriate decision-maker.
As Bob Graham summarizes, "The real test of market research is whether management is willing to use it for critical decisions. When sales people are showing large circulation numbers but offering huge discounts, you suspect that management knows something it is not telling in the published data. We use the same market research data in our marketing decisions that we provide to our customers. We believe our research is the best available and think that the success of our products is a strong testament to that."
©1989 Bell Atlantic Corp. All rights reserved.