you're reading...

A cautionary tale about polls

There are more poll positions being taken than even Jimmie Johnson could handle right now.  With campaigns offering a variety of takes on polls, I thought I should give you, fair readers, a quick cautionary tale about polls.

A poll is a delicate instrument.  Whenever you want to gauge the public’s opinion, asking 500 people to speak for a million is risky.  For a long time, telephone-based polling was very reliable.  However, the last ten years have seen technology confound pollsters.  No matter who’s polling today, the results are more in doubt than they have been for a long time.  The main reason?  Polling agencies buy calling lists compiled from land-line telephone numbers.  When land line phones are ubiquitous, there’s no problem: everyone has the same chance of being polled. But when more people are going without land lines every day, the number of people able to be polled over the phone shrinks.  Not just young people, although they are the ones most ignored in land line polling.  More and more people are dropping land lines in favor of cell phone-only coverage or voice over internet (VOIP) services.  When you go VOIP or cell-only, your number will not land in a pollster’s bank and therefore you won’t be able to participate.

Add to that the problem of caller ID: most polling agencies use computerized auto-dialers that will likely show up on caller ID as “Unknown Caller” or “Private Number.”  In my household, that’s code for “telemarketer: let the voicemail get it.”  People can avoid polls by not picking up the phone which many people do.  So polls are likely conducted among the people with landlines who don’t mind talking to telemarketers.  The pool of those people is getting constantly smaller, and means we are more likely to get an unrepresentative sample.  When the sample doesn’t represent the larger public’s views, the polls will be shown wrong in voting results.

Today’s Politico has a great story about the Arkansas and California Senate races that makes the point sharper.  Polling firm Research2000 stands accused of falsifying poll results to make challengers Bill Halter and Tom Campbell look stronger in their contests than they actually were.  In Arkansas, the problem was particularly acute because there were no other polls being conducted for public consumption, so the polls basically defined news coverage of the campaign.  Just like in the First Congressional District here, where the SurveyUSA polls have been the frame in which we’ve discussed the race.  If SurveyUSA’s method was wrong, as at least two campaigns believe it was, then we may have some shocking results in August.

The more people (and demographic groups) we exclude, the less likely that 500 person sample represents the views of the general public. We look to polls because of their history of being generally right (election night exit polls excluded) and the absence of other information.  The First District race is an example: the only objective cues reporters have had to access are fundraising numbers and polls.  So those two factors dominate the news coverage.  For all we know, those polls could be bunk.

What does this all mean?  Take these polls with a grain of salt.  Or better yet, a block of salt.  Polls today can show as divergent results as Jerry Moran’s poll that his campaign had him up 26% on Todd Tiahrt, or the new poll commissioned by Tiahrt’s campaign reported on here that shows Tiahrt and Moran at a dead heat.  Competing polls give us at least a bit more to work with, thankfully.  In California, Research2000’s poll was an outlier compared with others, so there was built-in skepticism.  There have been three polls of note in the Senate GOP primary here in Kansas: one each commissioned by the campaigns and the recent SurveyUSA poll that showed Moran up 20 points.  Who to believe?  The campaign-commissioned polls have to be the most suspect, calling both Moran’s and Tiahrt’s results into question. Polls conducted for campaign organizations are often rigged – we call them push polls.   There is no evidence either the Moran or Tiahrt campaign push polled anyone.  Campaign propaganda that masquerades as a poll is still propaganda.  Polls are supposed to be neutral instruments, but they are easy to manipulate and ‘push’ the respondent on the phone to answer in a way the campaign wants.

We must be skeptical and critical of polls because they are far from perfect instruments.  Consumer beware.


About Chapman Rackaway

Associate Professor of Political Science at Fort Hays State University. Hays Daily News and Insight Kansas columnist. Co-author of American Government: Politics and Political Culture (5th Edition).


No comments yet.

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: