This is an edited excerpt from a chapter in the book Research to Riche$: The Secret Rules of Successful Marketing by Jim Nelems, founder and retired CEO of Atlanta-based research firm Marketing Workshop. For more information about the book visit https://www.amazon.com/Research-Riches-Secret-Successful-Marketing/dp/156352709X.

Marketing research is often a catchall category whenever anyone is referring to collecting and reporting facts, opinions and attitudes. But just because someone calls something marketing research does not mean that it really is marketing research – or valid marketing research, which is the critical issue. 

Self-selecting samples

Back in the early 2000s, it was common for TV stations to post two phone numbers on the screen and say dial this number if you are in favor of the question and that number if you are against it. In the course of an evening, they might have 23,000 people who call in and vote their opinion, usually a simple yes or no. And how can 23,000 answers be wrong? After all, most polls only survey 900 to 1,000 people or less.

In most cases, the 23,000 people are wrong, as far as projecting the data. Yes, there are 23,000 people who expressed their opinion (say, 18,000 said yes and 5,000 said no). But this is a self-selecting sample. 

When conducting a self-selecting sample, only those who feel strongly enough about the issue take the time and effort to call. And in virtually every case, the people who feel strongest are the ones responding. The best example of this goes back to the early days of call-in polls, where then-Alabama Governor George Wallace won every call-in poll ever conducted for president that year. Wallace had a lot of friends and supporters, but they did not represent a cross-section of the voting public. 

Not scientific

Many polling opportunities posted on the Internet and available to the casual Web surfer are not marketing research. Many organizations have simply replaced the call-in telephone poll with questions on the Internet, available to anyone who happens to access their Web site (or link). Such polls create a false conclusion as to what the general public thinks. Just because someone can post a survey question on a Web site does not mean that they should, or that it is a representative sample of opinions. It may be entertainment but it is not scientific research. 

It is possible to rig an Internet poll through electronic ballot stuffing. Years ago, as reported in the New York Times, a particular photo contest, which had been receiving about 50,000 votes per week, suddenly received 5 million votes, most of which turned out to be in a different pattern from earlier votes. The contest was suspended when it was learned that e-mails from partisans had gone out encouraging people to vote a particular way, to make a political statement. 

Legitimate Internet polls 

It is important, however, to differentiate between Internet polling as described above and truly legitimate polling over the Internet through the use of panels and other opt-in polling opportunities. There are hundreds and thousands of such legitimate polls, in which respondents have been recruited to become members of Internet panels and agree to answer questions from time to time. For example, college students who have agreed to be a part of a panel might be e-mailed questions about their product preferences while away at school; a service provider may post questions on their Web site regarding a purchased product or service; a state tourism commission may post questions to those hitting their Web site to find out what types of activities the potential visitor is interested in, perhaps to know what type of information to provide. As long as the user of these polls understands their limits and projectability, and uses them for the purpose they were intended, so be it. 

Just because someone asks questions, or collects data, does not mean it is real marketing research.