Editor’s note: Jon Griffin is account manager at marketing research and analytics firm Optimization Group, Grand Rapids, Mich. This is an edited version of a post that originally appeared here under the title, “Online product design testing – a new application for max-diff.”
Online testing of marketing concepts is now an established practice. Numerous methodologies are used to test copy points, features and benefits. One methodology, max-diff, is becoming one of the most useful and straight-forward tools available today. The key characteristic of a max-diff design is that it forces respondents to choose between alternatives, which may be very similar and have a very similar appeal. Forcing respondents to choose the most preferred and least preferred item from a set of alternatives results in a greater ability to analytically discriminate between the alternatives than is possible using traditional monadic rating questions.
Typically, max-diff research relies on descriptive elements (such as features and copy points) to define the most motivating marketing messages. Now, marketers and researchers are utilizing the max-diff approach with product visuals, including product concepts, packaging, logos and more. With max-diff, respondents are exposed to a set of images and are asked to select the best (most appealing, most important, etc.) and worst (least appealing, least important, etc.). Respondents typically complete several scenarios where each scenario contains a different subset of images. The scenarios are carefully created using an experimental design that ensures that each item is shown an equal number of times and in different order.
Here’s an example of how this works for product packaging. Respondents are shown a series of four packaging concepts at a time, and are asked to indicate which package they most prefer and which they least prefer.
It’s a straight forward exercise that allows the respondent to assess alternatives quickly and react in a very intuitive manner. They don’t need to overthink details or explain their choices … they simply react.
The results from this max-diff exercise provides a score for each of the alternative images placed on a 0 to 100 point scale, indicating the likelihood that a particular style would be selected as “preferred.” In other words, each element “score” is the probability that the element is most preferred out of all of the elements that were tested.
So, why use max-diff for packaging and product concept testing instead of standard rating scales? Because research has shown that max-diff scores demonstrate greater discrimination among items and between respondents on the items. The max-diff question is simple to understand, so respondents from children to adults with a variety of educational and cultural backgrounds can provide reliable data. Since respondents make choices rather than expressing strength of preference using some numeric scale, there is no opportunity for scale use bias. Sometimes a simple approach can be the most powerful.