Editor’s note: Joe Hopper is president of Chicago-based Versta Research. This is an edited version of a post that originally appeared here under the title, “Why bigger is better for numeric rating scales.” 

If you grew up in the United States, you probably think big numbers are better when it comes to rating things. Higher scores on school exams are better. Higher scores in games and sports are usually better. Higher credit scores are better. Five-star restaurants are definitely better than one-star restaurants. In Germany it is often the reverse. Lower grade point averages, for example, are superior to higher ones.

This has important implications for how you should design research if you use numeric rating scales. People can use rating scales that are reversed from what they are used to. But their sensitivity as measured by a reversed scale will be dampened.

Here is what a new study just published in the Journal of Consumer Research found:

Numerical association[s] that people learn over time becomes part of their implicit memory – the type of memory that influences judgment without conscious awareness … This numerical association in implicit memory can then interfere with people’s ability to make evaluations using a newly learned format with opposite rating polarity, resulting in judgments that are less sensitive to numeric differences in quality level.

In multiple experiments, the researchers from Dartmouth College, Cornell University and the University of Michigan showed that the effect is robust and consistent over all kinds of evaluations, including bidding behavior, visual perception, purchase intent and willingness to pay. The research also found this to be consistent across different types of response alternatives, including open-ended, horizontal vs. vertical layouts, binary measures, scale measures and so on.

The implication is that generally you should design rating scales with numeric associations consistent with your audience. In the U.S. that means making higher numbers correspond with “better” and lower numbers correspond with “worse.”