Major markets – major mistakes?
Editor’s note: Carl Iseman is president of Assistance in Marketing, Inc., a Baltimore, Md., qualitative and quantitative data collection firm.
The "value" of a market to the researcher rests on a sliding scale. Depending on the research objectives and information needs, cities are selected according to the likelihood that they contain adequate samples of the sought-after population. BDI’s, CDI’s, demographic profiles, census data, are all used to substantiate the use of one city or market over another.
On its face, this seems a perfectly legitimate approach because the presumption is that all the facts have been considered and given the appropriate weight. But sometimes, behind all that empirical wisdom, lurks the very unscientific and substantially unsound, "preconceived notion."
Pre-existing opinions about markets aren’t new but they’re not always an accurate or suitable compass for quality research. Quality research requires that there be a match between the sample taken and the characteristics of the market surveyed. This means that the characteristics of a market have to be known and routinely reviewed and updated.
Unfortunately, this is not always the case. In fact, I experience this frequently at trade shows and professional research conferences and I see it in very practical terms as I attempt to market my own facility: corporate researchers and full-service firms opt for major markets like New York, Los Angeles and Chicago because they feel sure that they will get a nationally representative sample of a population with their desired characteristics.
Enough needles
The prevailing logic (it seems) is that if the haystack is big enough there will be enough needles. However, the rationale upon which the use of these markets has traditionally been based isn’t as valid as it once was. It isn’t good enough any more that the haystack be big enough to generate a lot of "needles." The haystack also has to have the right mix of values and lifestyles because, as Judith Waldrop has written, it is these critical features of a market that "predict consumer behavior better than demographics." ("Markets With Attitude," American Demographics, July 1994.)
In her article, Waldrop recounts a telephone conversation she had with the representative of a major network newscaster who had contacted the magazine to recommend a city where the demographic profile was most representative of national averages. She recommended Tulsa, citing the magazine’s national profile studies based on census data. The representative then wondered about Indianapolis and Waldrop told them that Indianapolis was among the Top 20 in national profile, but that the city of Tulsa ranked No. 1. The newscaster’s representative thought for a moment and then replied, "Can you tell me more about Indianapolis?"
This scenario is all too common. Despite demographic and psychographic information to suggest otherwise, Indianapolis was probably the market surveyed for the newscaster’s office because they probably found it unbelievable that a town in Oklahoma would have more all-American attitudes and be closer to a national representative market than would Indiana’s heartland capitol. Or, that it would be easier or more convenient to travel to Indianapolis.
Humorous though it may be, Waldrop’s anecdote provides a wonderful example of how a preconceived notion can devalue a study -- making it less representative and, more importantly, less accurate and actionable. I have a lot of experience with preconceived notions and feel well-qualified to speak to the issue. As the owner of a data collection firm in the Baltimore metropolitan area, my company has felt their effect.
Beer and baseball
Over the years, I have come to regard Baltimore as the Rodney Dangerfield of research markets because we seem to command such little respect. Corporate marketers and research project managers have widely regarded Baltimore as a city whose population is too blue-collar, too Democratic, too lower middle-income, and whose people are under-educated and too committed to beer drinking and baseball. They seem to view Baltimore as a dingy little port city with no culture to speak of that’s located too close to more sophisticated markets like Washington, D.C., Philadelphia, and Wilmington, Del. None of these preconceived notions are true.
Boasting over 2.5 million residents, Baltimore is one of the largest metropolitan areas in the U.S. - a big haystack in its own right. According to the 1997-98 findings of the Maryland Department of Business and Economic Development, Baltimore’s per capita income is the fifth highest in the United States. Of Maryland’s population (age 25 and over) 32.5 percent hold a bachelor’s degree or above. This is the highest percentage in the nation! Baltimore also boasts one of the highest concentrations of scientists and engineers; some of the nation’s finest hospitals and health care institutions; and one of the nation’s highest concentrations of computer service and emerging communications technologies professionals.
In Waldrop’s article, to determine which cities best matched the national profile, the magazine built an index "of dissimilarity to compare the national percentages of households in each of these VALS2 segments with the percentages of each metropolitan area. The index value indicates the share of households that would have to change categories in order for the metropolitan area to perfectly match the national distribution."
The index for Baltimore was 1.95. This means that fewer than 2 percent of Baltimore households would have to alter their lifestyles and values in order to make Baltimore perfectly representative of the U.S. as a whole. And guess what: This is as near to ideal as you get. Guess what else: New York, Chicago and Los Angeles were not among the Top 25 metropolitan areas identified on the list. Baltimore; Wilmington, Del.; Brazoria, Texas; Richmond-Petersburg, Va.; and Evansville, Ind. round out the Top 5 "Well Balanced American Metros."
Garnering more respect
So, armed with this wonderful information, and shouting it from the rooftops, you would think that Baltimore (and other markets in a similar predicament) would surely be garnering more respect (and more visitations) from the corporate and full-service market research communities. But they’re not. You would think that qualitative researchers would be actively evaluating more "virgin" territories, offering fresh respondents, in fresh markets for their focus groups. But they’re not.
One reason may be that the list of so-called tried and true research markets - that is, markets that have historically yielded good, reliable, projectable results - are no longer necessarily markets representative of the U.S. population. Researchers seem to go back to them, however, out of comfort, like an old shoe. They don’t seem to want to waste valuable time re-evaluating markets that have served them well in the past. They’re comfortable with the facilities; they know the cities, restaurants, attractions. Besides, new markets are such a hassle. They don’t know what to expect, who to work with, etc. It can be a lot of work to challenge those pesky preconceived notions.
Clients have notions too
And it’s not just researchers, it seems that their clients also have a few notions of their own. I routinely hear that our clients’ clients dictate not only the general geographies, but also the specific cities in which data is collected. The clients are telling the research firms where they need to go in order to find large populations of the target sample. They often base these decisions on buying indexes, sales data, and other primary and secondary resources and, in some instances, this information is just not available. When it isn’t available, clients tend to fall back on their own opinions. For example, Pacific Bell isn’t likely to conduct market research in Miami to find out how well its customer service is being perceived. Absent this kind of obvious circumstance, I think researchers have to ask themselves to what extent they are serving their clients by not insisting that updated and objective criteria be applied to market selection.
Sometimes, I wonder if professionally trained market researchers are unable to guide and educate their clients according to the research objectives because of the clients’ preconceived notions. Could it be that we are not challenging our clients - allowing them to opt for markets that are less than optimal - because we, as businesses, are wary of questioning their choice? It is, after all, the quality of the data and the suitability of the market in which it was collected that is paramount. We shouldn’t compromise on this. Never.
Limit investigations
Preconceived notions about markets will compromise data because these notions shrink the research universe and limit investigations into the true suitability of an old market over a new one. New York, Los Angeles and Chicago are excellent research markets. However, there are other fine markets that are rated as far more desirable, All-American venues in which to do research.
Of course, I’d like to see more researchers in Baltimore. But if I don’t, let it be because our market was realistically reviewed and discarded for reasons relevant to the study and not, as is my great fear, for preconceived notions and a general lack of awareness. I welcome your suggestions and comments on my hypothesis.