Editor’s note: Stephen J. Hellebusch is president of Hellebusch Research and Consulting Inc., Cincinnati.

In the book Predictably Irrational by Dan Ariely, a study is described that reveals something very interesting about the effect of context on how we make decisions. Knowledgeable and very intelligent people who were fully aware that certain numbers were chosen in a completely arbitrary, random fashion were still influenced by those numbers when making decisions about how much they would pay for certain items. This is described as arbitrary coherence, a term found in the behavioral economics literature. This is so illogical, and potentially so important for marketing research on pricing, that I replicated it to see for myself, as described in an article (“Under the influence”) that appeared in the May 2010 issue of Quirk’s.

In the original study, MIT students were asked three questions about several items that might be purchased in a store. However, the first step they were asked to take was irrelevant: to write the last two digits of their Social Security number (SSN) in dollars ($) next to the item. So, if their SSN ended in 12, they wrote $12. If it ended in 87, they wrote $87. Next, they were asked to indicate whether they would pay that amount for the item (yes/no). Finally, they were told to pretend the item was being auctioned, and to write in the amount they would be willing to pay. As the SSN$ went up, so did the amount the students were willing to pay.

In my prior study, in May 2009, 71 associates at Directions Research Inc., Cincinnati, graciously completed a task with each of four items: a 12-bar value-pack of Irish Spring soap; one pair of Dr. Scholl’s hand-stitched leather loafers; a 750ml bottle of Baileys Irish cream; and a Teleflora Spring Pitcher floral display. With a few reservations, the replication demonstrated that arbitrary coherence worked. For an item of low interest, Baileys Irish cream, there was a very weak arbitrary coherence effect. For an inexpensive item that everyone knows is inexpensive, the Irish Spring 12-bar pack, it did not work at all.

There are many studies that could be undertaken to define the limits of arbitrary coherence and how it might impact decisions we make in designing research, not to mention decisions that our respondents make. The one investigated here is, “What happens if no decision is made about the Social Security Number dollar amount - if the arbitrary number is just next to the place where respondents enter what they are willing to pay?”

Objective
Determine what occurs if people just write the SSN$ amount and are not questioned as to whether they would buy the item for that amount. Does the arbitrary coherence effect vanish, or is it robust enough to endure even if the “middle” decision is eliminated?

Once again, the good people of Directions Research Inc. helped find out.

Method
Eighty-four associates participated in this research in April 2010. The method was internal online interviewing. Due to the need to compare the three-question effect with a possible two-question effect, it was necessary to split the group somehow. Each person was asked their year of birth, and all odd-numbered-year people were assigned to the 3Q cell - a three-question cell; all even-numbered-year people were assigned to the 2Q cell, a two-question cell. For no reason other than chance, 42 of each responded, so there are 42 in the 3Q cell and 42 in the 2Q cell. These are mostly highly-educated and research-savvy people, much like in my original experiment. Plus, many of them had completed the task before, about a year prior.

Four items were again chosen at random. They are: a crocodile-skin wallet, a necklace of pearls from Tahiti, a copper cooking bowl and The Complete Works of Lewis Carroll.

Using judgment, it was thought that none of these items would be of a known, inexpensive cost and it was hoped that all would be of reasonably high level of interest to people.

For each item, the 3Q cell is asked:

Please enter the last two digits of your Social Security number as if it were a price in dollars.

Would you be willing to pay the above amount for a ________?

Pretend that a ________ was being auctioned and you were participating in the auction. In whole dollars, what is the maximum amount you would pay for a ________?

In the 2Q cell:

Please enter the last two digits of your Social Security number as if it were a price in dollars.

Pretend that a ________ was being auctioned and you were participating in the auction. In whole dollars, what is the maximum amount you would pay for a ________?

Results
The study reported in the earlier article showed results implying that there had to be a reasonable level of interest in the products to see an arbitrary coherence effect, e.g., there was only a weak effect with the Baileys Irish cream, which relatively few wanted to buy at all.

It is only in the 3Q cell that the overall level of interest can be examined (because question two was not asked in the other cell). Less than half were interested in the wallet, bowl or book (Table 1).

The arbitrary coherence effect was not replicated in the 3Q cell. There is either a mild effect or no effect of Social Security price on the price willing to pay for the necklace, bowl and book. For the wallet, the effect is, if anything, reversed - the higher the SSN$, the less people were willing to pay (Table 2).

Similarly, no effect was found with only two questions for the wallet, necklace and bowl. In this case, the book exhibited a nearly reverse effect (Table 3).

Since it is starting to appear as if the level of interest is a major factor influencing the arbitrary coherence effect, those who said they would buy for the SSN$ in the 3Q cell were examined separately. The base sizes are small, but the effect does show up under these conditions. The correlations were tested against zero, and only the bowl (0.65) is statistically significantly above zero at the 90 percent confidence level. Among those interested, the others trend in the right direction, at least (Tables 4.1 and 4.2).

Discussion
This research failed to replicate the arbitrary coherence effect, which appears to be somewhat temperamental. There are several possible reasons for the failure. Perhaps the fact that many of those who participated had participated in a similar study one year ago and had seen a presentation of those results created more-aware respondents, who were not so easily influenced. Since, by definition, the effect is supposed to occur even when respondents are aware that the first number entered is arbitrarily chosen, this seems unlikely.

The level of interest in the products appears to be a key component - high levels enhance or reveal the effect, low levels vaporize it. In that case, the advice for marketing researchers would be to ask a question to determine level of interest without price prior to asking about price, so the interested group can be broken out.

The original question - do you need the second question concerning purchase for the SSN$ in order to see the arbitrary coherence effect - remains unanswered. Since there was no effect when there should have been, seeing no effect in the 2Q cell is inconclusive.

Be careful about the order

After being intrigued by the arbitrary coherence effect and spending considerable time analyzing data related to it, I am convinced that 1) it exists, and marketing researchers have to be careful about the order and placement of pricing questions lest it distort results and 2) it is a weak effect, easily eliminated through care in research design.