Lessons from Florida
Editor’s note: Dick McCullough is president of Macro Consulting, Inc., a Palo Alto, Calif., research firm.
In their recent battle to be president of the United States, George W. Bush and Al Gore, Jr. virtually ran a dead heat. And it took the system an agonizing month and a half to sort it all out, globally showcasing a substantial amount of dirty laundry in the process. What went wrong? Who really won? How can we keep it from happening again?
The answers to these questions don’t come from the law. They don’t come from politics. Or even the U.S. Constitution. They come from market research.
Think of the presidential election as a really big market research study. Any good researcher will tell you that the first step in conducting a successful market research project is to define your research objective in clear, measurable and actionable terms and then get the entire project team to understand and agree to the objective.
What was the objective here? Before the election, although never explicitly stated, most people would probably have said the objective of our little research project (the national election) was to choose a president. But it turns out, this objective is not necessarily clear or measurable, although it has proven to be actionable in the extreme. After the election, the objective that each candidate implicitly assumed shifted subtly. Bush’s lawyers based their arguments on the assumption that the objective was to win the election. In their arguments, Gore’s lawyers assumed the objective was to discern the will of the people. Apples to oranges. Politics.
If the objective was to win the election, then the issue is simply who got the most votes within the rules of the game laid out prior to November 7. And Bush wins. Winning the election is clear, measurable and actionable. Also technical, rigid and perhaps irrelevant. What if the person winning the election is not the person most people wanted to be president? Are we happy with that? If not, and if we say we want the objective to be to discern the will of the people and we also want the rules of the election to be such that the will of the people is discerned, then we are faced with some classic market research issues.
The first issue is sampling error. Only about half of eligible voters actually vote in any election. Does that half accurately reflect the will of the other half? Probably not. It is often said low turnout favors the Republicans and high turnout favors the Democrats. You may say the voting half doesn’t need to reflect the will of the non-voting half. That if someone chooses not to vote, that’s his or her problem. Okay with me. But you’ve now changed your objective to discerning the will of the people who voted. And that’s the Gore view. The Bush view is one step farther out: their objective is to discern the will of the people who voted correctly (they would say “legally”). And we’re back to arguing about objectives (see how important it is to get that straight at the beginning?).
But the really big issue is measurement error. Measurement error is the difference between what the voter meant to do and what he actually did. So if I wanted to vote for Gore but I actually voted for Buchanan, that would be measurement error. If I wanted to vote for Gore but I actually failed to punch out a chad completely and was officially counted as a no-vote, that would be measurement error.
In an election, as in any research project, there are two types of measurement error: random and systematic. A random error affects all votes with equal probability and, therefore, would not affect the outcome. That is, it wouldn’t affect one candidate more than the other. If all voters voted in exactly the same manner, say the old punch card system, all voter punch cards were handled in the same way and to the same degree, and all votes were counted in the same machine (or at least in exactly the same way), there would still be errors in the counting. But those errors would be randomly distributed across the two candidates. The winning candidate would be extremely likely to reflect the will of the people (at least the people who voted).
But if there are differences in the way the voters vote (punch card vs. optical scanner), or the way the votes are counted (machine vs. hand), then the error terms are no longer random and equally distributed. These new error terms could favor one candidate over the other. For example, if Bush supporters more often voted using procedures yielding fewer no-votes than procedures that Gore voters used, there could be an error favoring Bush. Then Bush could win the election but not reflect the true choice of the people. Another example: if no-votes are hand counted by different people using different criteria, say Broward County vis-à-vis Palm Beach, then another systematic error could occur. And of course, if no-votes are counted in some counties and not others, then once again a systematic error term would have been introduced.
A brief sidebar: since there has been no substantive claim of any fraudulent or intentionally malicious behavior by either side, I will ignore dishonesty as an error source. Same for system or machine malfunction.
I haven’t yet mentioned exogenous effects such as the networks calling Florida for Gore before precincts in the Panhandle had closed, thereby discouraging up to 20,000 Bush supporters from voting at all. Or military absentee ballots where the military, not the voter, failed to get a postmark on the ballot, thus disqualifying an otherwise valid measure of voter intent. But these are just other examples of systematic error. Even inclement weather could introduce a biasing effect. Bad weather could be a source of systematic error if the weather affected voter turnout only in a predominantly pro-Gore or pro-Bush geographic area. It would be a source of random error if it affected pro-Gore and pro-Bush voter turnout equally. I’m sure you can imagine dozens of scenarios containing either or both systematic and random errors.
Those of us who collect data for a living, be it in market research or any of a number of other fields, know there is always, always error in the data set. The key to getting the right answer is not eliminating all error. That is impossible. It is minimizing non-random or systematic error. Random error generally won’t mislead you, especially with a large sample size. Systematic error is much more likely to.
So what do we do here? Ideally, we would collect every vote in exactly the same way. And going forward, it’s obvious that we need to substantially revamp our voting procedures nationwide so that everyone within a given state votes in the same way as everyone else in that state. Fairness demands it.
But what should we have done with this election? With this data? Once the data is collected, it’s too late to change the data collection procedure. The cow is out of the barn, as my grandfather would say.
In market research, we recognize imperfections exist in data. We don’t generally throw out the study because of it. We first design studies with minimal systematic error. We next ensure that there is no question of fraudulent data. We assume the random error doesn’t affect the results. Next, we make every effort to clean and edit the data set to insure it is as accurate and as complete as possible. That would be analogous to doing hand counts (but everywhere and in exactly the same way). And finally, forced by practical realities, we assume, given an absence of malicious intent or mechanical malfunction, that any random or systematic errors that remained occurred equally often to both sides and cancelled each other out. At this point, we let the chips fall where they may.
Too bad politicians aren’t researchers.