Editor's note: Jonathan E. Brill, Ph.D., is principal of Next Generation Research, Solon, Ohio.
Today more and more organizations are conducting attitude research projects such as customer satisfaction surveys, advertising concept studies, and corporate image surveys. This is both good and bad. It's good because, when properly implemented, an ongoing program that measures consumer or business market perceptions can help organizations be more competitive and profitable. But it's bad because such programs rely on accurate assessments of attitudes, and typically there is far too little accuracy in today's attitude surveys. Symptomatic of the dearth of accuracy is the widespread use of single-item survey measures.
Accurate attitude assessment through survey research requires that attitude measures be reliable and valid. A survey measure is said to be reliable if its observed value or score can be replicated with little variance - that its observed value can be relied upon, so to speak. A measure's validity is established when there is strong evidence that the measure represents what it is intended to represent exactly - nothing less and nothing more. In attitude research, such measurement objectives can rarely be realized through use of a single item in a survey.
Validity compromised
Validity is almost always compromised through the use of single-item measures for the simple reason that very few phenomena of interest evince only single perceptual aspects. Thus it typically is impossible to capture the entire domain of the phenomenon of interest in just one question.
Consider, for example, measuring satisfaction with a restaurant. Because there are several (obvious) dimensions of a restaurant that may contribute to a customer's satisfaction - e.g., food, service, ambiance, price - a single-item measure such as, "How satisfied are you with this restaurant?" (accompanied by an appropriate rating scale) will not suffice. It is simply not reasonable to presume that all respondents will consider all four of these aspects, only these four aspects, and each of these four aspects with identical degrees of importance when formulating their satisfaction ratings.
The best solution is not, as examination of most satisfaction surveys might lead one to believe, to ask about each of these four (hypothesized) dimensions separately with a single question for each. There are two salient reasons for this.
The first is, once again, a validity issue. Returning to the restaurant example, it should be clear that using a single item for each hypothesized dimension (i.e., food, service, ambiance and price) suffers from precisely the same problem that "How satisfied are you with this restaurant?" does: Surely each hypothesized dimension is reflected by several aspects.
For example, potential aspects of one's evaluation of the hypothesized service dimension include (but are not necessarily limited to) attentiveness of the server, the promptness of the server in responding to requests, the promptness of the host or hostess in greeting patrons, the courtesy shown by the host or hostess in greeting patrons, the efficiency with which the host or hostess seats patrons, the general courtesy exhibited by the server, the general courtesy exhibited by other staff (such as buspersons), the general helpfulness shown by other staff, and the accuracy of the food order.
The second reason asking about each of the four hypothesized dimensions is not the solution is related to reliability. Single-item measures are widely recognized to be inherently less reliable than measures using multiple items because of the potential for error associated with survey administration and the recording of the respondent's evaluation.
Respondents may be moody or elated at the time of survey administration, which may lead to overly negative or positive ratings. Alternatively, boredom may cause respondents to become lazy as they consider their responses which, in turn, may lead to the recording of unintended or otherwise inaccurate evaluations.
In contrast, when the ratings of related (i.e., intercorrelated) multiple items are added together to form a single measure, such errors tend to average out. Thus, these multi-item measures, commonly referred to as "scales," offer greater stability or consistency in the summed rating values than in the value of any individual item. This means that the reliability of single-item measures can always be improved through the addition of a second, strongly correlated item.
Smaller error
A further advantage of multi-item scales over single-item measures is that, for any given sample size, the mean value of a multi-item scale will have a smaller standard error than the mean value of any individual item (i.e., any single-item measure). Astute managers will not overlook this point, because it means that the use of multi-item scales, rather than single-item measures, has the potential to reduce research expenditures considerably.
Smaller standard errors provide greater statistical power, thus allowing the use of smaller sample sizes. Reducing the required sample size, perhaps from 400 to 200, with no loss of statistical power, can more than compensate for the added cost of developing and using multi-item scales instead of single-item measures.
In cases of tracking studies where a survey is repeated again and again - commonplace for corporate image surveys, advertising effectiveness studies, customer satisfaction research, and other total quality management survey programs - the cost savings can be huge. Best of all, the validity and reliability of the data will have been greatly enhanced which, in turn, empowers management to make decisions with increased confidence in the accuracy of the research findings.
Rip-off
For these reasons, single-item attitude measures represent the biggest rip-off in survey research. Skilled researchers know this and try to avoid using them. In my opinion, research companies that develop questionnaires without multi-item scales for measuring attitudes are either ignorant, lacking the statistical expertise necessary to construct and evaluate the adequacy of multi-item scales, or not concerned about the reliability and validity of survey findings. Regardless of the reason, these research companies cost their clients more over the long haul in terms of research dollars and profitability of business decisions.
In the face of today's increasingly competitive business environment and ever-tightening budgets, responsible managers must aspire to realize greater value from their research dollars. Shrewd managers will respond to this challenge by insisting on the use of survey research organizations that are truly capable of delivering the high quality attitude measurement that today's business climate demands.