Borrowing from one to enrich the other
Editor's note: Alan Kornheiser is a principal and research director of Sophisticated Market Research, North Salem, N.Y.
Once-academic techniques, such as conjoint analysis and multidimensional scaling, have become increasingly common in everyday quantitative market research. Today, it is almost the exceptional study that does not include at least a quadrant analysis or a set of factor scores in its report, even if the results simplify the reality beyond recognition and force the data into a Procrustean bed of limited dimensions.
Given that such quantitative techniques, especially when improperly applied, brush ambiguities and the small but telling detail under the rug, it may seem strange that we are proposing a variant of their use in qualitative research. After all, the purpose of good qualitative research is not to simplify but to enrich; not to reduce the number of key variables but rather to develop hypotheses and generate as wide a range of possibilities as possible. However, if we focus not on the underlying mathematics of such quantitative techniques, which are indeed designed to simplify, but focus instead on the test methodologies themselves, we may find ourselves with new and useful methods for generating ideas, terminologies, and relationships in a qualitative environment.
Accordingly, we have adapted three multivariate techniques - conjoint analysis, cluster analysis, and multidimensional scaling - for use in qualitative research. While we do not employ mathematical reductions of the results, we do use the sorting, trade-off, and scaling procedures inherent in these methodologies as the basis for rich idea and hypotheses generation. What follows shows how we do this, and why.
Pseudoconjoint analysis
In a quantitative study, conjoint analysis is typically used to determine underlying valuations. While a respondent may say, and believe, that he considers price, a range of features, and quality to be equivalently valuable, in fact he will invariably choose to trade one off for another at different rates. For example, price and reliability are vitally important in choosing an automobile. Different respondents will choose different trade-offs; one will be much more price sensitive, another far more concerned with quality. A well-designed study can have prospective car buyers trading off price, quality, features, attractiveness, dealer service, and many other variables in such a way as to effectively model a consumer's buying decisions. By presenting the consumer with a deck of options (i.e., a set of cards, each containing a different set of car descriptions) and asking him to rank order the deck in terms of desirability, a skilled researcher can determine why a prospective buyer makes the decisions he makes, even if the buyer himself cannot clearly express the trade-offs.
As qualitative researchers, we are interested in understanding precisely what this technique deliberately ignores: why the trade-offs are made. While conjoint analysis argues that it can predict buying behavior without explicitly letting buyers describe the reasons for that behavior, we as qualitative researchers are most interested in precisely those reasons...and much less interested in making predictions. Accordingly, if we turn conjoint analysis on its head, we may find we can use its tools as a means of learning why decisions are made, without actually trying to predict the decisions themselves.
This technique, pseudoconjoint analysis, is best designed to generate understanding of the way choices are made. We begin in the same place standard conjoint analysis begins: with a deck of options. Since this is being done in a group setting - although it works just as well in minigroups or even in-depth interviews - a far smaller set of cards is used: six is typical, although one might use as many as a dozen if many variables were being examined. By way of contrast, true conjoint analyses typically use dozens of cards at a minimum.
It is vitally important that this set of cards expresses real, complex choices. While each card need not contain all possible options (for example, one card might not discuss a car's color, while another might simply omit the issue of reliability), the entire deck must include all options, and it must include them in such a way as to require respondents to consider real trade-offs; there is no point in having people decide they'd rather buy a cheap, reliable blue car than an expensive, unreliable red car.
Respondents are then asked to sort the cards, from most desirable to least desirable. When done in a group session, as is most common, the moderator tries to obtain a consensus - which happens, more often than not, especially if only a limited number of cards are used. However, almost as commonly no consensus can be reached and there will be disagreement, as one respondent prefers this while another prefers that. This is actually the more desirable - and certainly the more realistic - outcome.
Where conflict arises, the moderator must generate discussion. Where is there disagreement? How important is this disagreement? Other than this disagreement, is there consensus? The heart of such discussion is the elucidation of the extent of differences in perceived importance of various elements and the reasons for these differences.
This is best done using standard laddering techniques. A difference over price/quality trade-offs might, for example, be explored by asking why price is more important? What does price mean to you in this context? What else? What does quality mean? What else? One takes the terms resulting and ladders them up. If quality means reliability, why is that important? If price means that you can afford other things, what other things? Why are they important? And so on.
This procedure is repeated again once a consistent set of choices has been generated. What about this choice makes it better than that one? What does such a choice mean to you? What does that mean? Again, and so on, using standard probes. By forcing decisions, by requiring respondents to set priorities, rich discussions about why choices are made and how choices are made invariably result.
A good example of how this process works involves a recent study conducted for an international airline that wished to improve its in-flight entertainment in its business and first-class cabins. Except for a (perfectly understandable) revulsion at the types of movies typically shown in airplanes and the usual complaints about air flight, several groups of business travelers were unable to generate any interesting discussions about their desires for in-flight entertainment. Worse, when shown a range of possible improvements, these frequent fliers liked all of them and were unable to explain why one was better than another. However, when presented with a series of possible sets of entertainment (e.g., individual movie screens and GameBoys vs. improved access to computer power supplies and a non-stop stream of snacks), the respondents were able to create very clear preferences and to discuss the reasons for their choices with great clarity. Distinct types emerged - workers vs. sleepers vs. players vs. self-entertainers - and the way in which travelers moved from one category to another during a flight also emerged. Note that the pseudoconjoint was valuable not because it enabled us to find and identify these groups; it was valuable because it catalyzed the discussions that led to these groupings, with their needs, preferences, and language.
By forcing preferences among fairly equivalently valuable combinations, we are able to create rich conversations where there might be only silence.
Pseudocluster analysis
In quantitative studies, cluster analysis is a general term used to describe several statistical techniques that group - as one might expect - similar things closely together. The technique can group all the products that appeal to young men over in this corner and the products that appeal to older women in the opposite corner. Because it contains some of the more basic simplification algorithms (and, in fairness, because it is often done in only two dimensions, which is almost guaranteed to wipe out any useful subtleties), cluster analysis is almost the direct opposite of good qualitative analysis. However, by borrowing not the analysis and not even (as above) the test materials of cluster analysis, but rather by reproducing cluster analyses outputs, a rich new way of generating discussion and deriving information is possible.
In pseudocluster analysis, the moderator simply places a large number (a dozen is often a useful number) of products on the table: a dozen types of candy or perfume or software or anything else being discussed. Respondents are asked to group them into as many sets as they feel appropriate. They then discuss the reasons for their groupings, what similar products have in common or different groups do not have in common.
To avoid trivial results, the moderator should feel free to make this harder for the respondents. If they initially group by color, forbid grouping by color. If they initially create three groups with everything interesting in a center group, have them do it again using only the products in the center group. Once they've created useful groups, forbid all the key discriminators they've used and have them do it again. Continue until you have generated a rich and complex vocabulary of how products differ and why.
You can then continue by asking where an ideal product would go on such a set of groupings. Or ask where a product for a young person or an old one or one who hated TV would fit. You can ladder from reasons for difference to reasons for choice, or from reasons for choice to reasons for difference. The only key is that you must keep laddering...each time a grouping becomes firm, probe to determine why that group exists, why the differences are important, and what those differences mean.
This is a remarkably simple exercise. Respondents greatly enjoy the tactile nature of actually maneuvering real products on the table, and good internal discussions (take notes - the recording will miss them!) during the grouping will provide additional richness.
By giving respondents tactile objects to organize, it becomes much easier for them to find and then discuss similarities and differences among the products.
Pseudo-MDS
The quantitative technique known as multidimensional scaling is an extremely good way to establish the key dimensions of variability when you have no sense of appropriate terminology. The moderator simply asks respondents to tell him how similar (or different) are any two pairs of objects, using many sorts of simple scales. This is repeated with all of the (or many of the) possible pairs of objects (which can be brands of cigarettes or types of blue jeans or makers of computers) and uses a sophisticated mathematical algorithm to generate the actual dimensions being used to discriminate. Interestingly, the first dimension is almost always "how much I like it," even for objects for which liking would seem to be irrelevant or fairly consistent.
This is a time-consuming process. One cannot, in a qualitative session, ask respondents to evaluate multiple pairs. However, one can ask respondents to do something simpler: put objects on a table, or on a wall, in such a way that the ones closest to each other are the most similar and those furthest away are the most different.
Clearly, this process - pseudo-MDS - is procedurally similar to pseudocluster analysis. However, in actual use the differences are profound. To begin with, it does not allow any clusters. All products must be placed distinctly. Secondarily, the distance between objects is important. While in pseudocluster analysis the final result is almost always three or four or six or 10 piles arranged neatly on the table, in pseudo-MDS the outcome consists of products scattered very widely, and with the distance from one to another being very important.
It's actually hard to do this on a table, and it works best with 3x5 cards fastened to a wall. However, a single wall allows only two dimensions, so one ideally lets respondents use several walls...and the table...and the floor...and even the ceiling!
The key probe in this technique is not "Why are these two different?" - since all objects are different. Rather, it is "Why are these two so much more different than those two?"
Here we are reinventing an ancient military expression: quantity has a quality all its own. Suppose, as an easy example, we are evaluating candy, and respondents have put a very sweet candy at one end of the room and a very tart one on the other end of the room. They will easily tell you that they're using sweet/tart as a way to divide the candies. However, it is an easy probe to ask why this simple dimension has become so very important...why have they not differentiated between cherry and chocolate that way, or between inexpensive and expensive? One obtains a key probe as to what it is in this dimension that matters so much, and one develops a richer vocabulary and sense of what matters in the category.
By failing to explicitly define what differences are important but forcing respondents to in some way find very explicit differences, pseudo-MDS exposes the underlying structure used to define a category.
Not true substitutes
These pseudoquantitative tools are not, as the reader surely now appreciates, in any way substitutes for true quantitative tools. Rather, they are a set of methodologies that borrow from quantitative analyses to give the interviewer new ways to make his respondents stop and think about the topic at hand. They are not magic and do not work automatically. However, when coupled with appropriate probing, follow-up, laddering, and the encouragement of group interaction, they offer a new way to provide understanding of the topic at hand.