Why Polling Doesn't Match Election Results

(iStockphoto)

This post was written by CBS News Director Of Surveys Sarah Dutton and CBS News Consultant Kathleen Frankovic.

The results of the most recent CBS News/New York Times Poll have been criticized –- by those who don't like its results. The criticism is that the answers to one question in the poll -- how did you vote in the 2008 election for president -- don't mirror the actual vote.

But that is nothing new. This poll and others have asked about past vote for many years, and the results rarely match the voting results.

Here's the most extreme example: The University of Michigan's National Election studies, arguably the best academic analyses of why Americans vote the way they do, found huge disparities in how Americans reported their 1960 vote in their 1962 and 1964 post-election studies. When Americans were asked just after the 1960 election how they voted, John Kennedy and Richard Nixon were tied in the poll (matching the results: Kennedy just eked out his 1960 victory). Two years later, Americans recalled giving Kennedy a comfortable win. In 1964, after Kennedy was assassinated, the difference was even more dramatic – Americans "remembered" that they had elected Kennedy by a two to one margin!

It goes in both directions – Democratic AND Republican: In January 2002, when George W. Bush's approval rating was 82 percent, Americans recalled that they had given him a 12-point victory over Al Gore in November 2000. If only they had! We would have been spared the 35 days of electoral wrangling that followed that election.

In the latest CBS News/New York Times Poll, Americans claim they gave Barack Obama a 48 percent to 25 percent victory over John McCain (he won by just seven points). Other polls also show an over-reporting of Obama's support: an L.A. Times Poll conducted in December showed a 20 point lead for Obama among voters.

Why does this happen? Two reasons. Many voters simply do not want to admit to a poll interviewer that they didn't vote in the election. Polls regularly show a significant overstatement of voting behavior: an April 2009 Gallup/USA Today poll showed 81 percent of Americans said they voted in the 2008 election. Actual turnout in the election has been estimated at about 57 percent of the voting age population.

Second, voters' memories are affected by their current opinions. If the elected President is particularly popular, the desire to remember voting for him increases. Barack Obama's approval ratings are in the 60s, both in the latest CBS News/New York Times Poll and most other polls. Voters may like to associate themselves with the winner (in their memories, at least), and many are reconciling their current opinions with how they voted last November.

Of course that means that if presidents become unpopular, memories will change, too. In June 2005, voters said they had supported John Kerry over George W. Bush by a point, 38% to 37%. (President Bush won re-election by three points.) President Bush's popularity was in the low 40's – and declining -- at the time. And there were moments during the Watergate scandal in 1974 when voters did not remember that they had elected Richard Nixon in a 20-point landslide (the margin of his victory over George McGovern shrunk along with his sinking approval rating).

Voters may also be affected by the questions they've already been asked in the poll. Topics that tend to favor Democrats, such as health care or the economy, could cause a higher Democratic response to the 2008 vote question, which is asked toward the end of CBS News Polls.

Why ask the question if it yields data that don't reflect the actual voting results? It is useful to pollsters as another way of understanding a president's popularity – and his weaknesses.

So, as most pollsters know, criticizing a poll's results on the basis of how people respond to this particular question makes no sense.

Mark Blumenthal, the editor and publisher of Pollster.com, addresses the issue on his blog here.

And Kathleen Frankovic wrote about respondents lying to pollsters in a 2007 column for CBSNews.com.

[schmidtyphi]

@earlysaid

"It doesn't take a genious to know that the GOP have become a pathetic group of people who have lost their soul and decency. They will never have the best interests of the America people in their thinking. They work for their uninformed and hateful base. "

There are uninformed and hateful people on both sides. To suggest otherwise -- in particular, that anyone who consistently aligns with the GOP (the base) is uninformed or hateful -- is just silly. It's also a display of profound ignorance and ad hominum hate, isn't it?

Makes one wonder if you're encumbered with the ideas of the non-hateful, well-informed intellectual conservative? I don't think the content of your post would occur to someone who was.

[Pojman]

Having read the poll data of the Health Care and Cap and Trade Polls recently done in their entirety, and being educated in polling processes and statistical analysis, I can state that the conclusions drawn appeared to be flawed in the following manner:

The polls ask questions of reflective of public opinions on each topic but the opinions the report writers promote for their articles in applying the results to legislation before Congress and the Executive branch is done by irrelevant conjecture.

In the health care survey, for example, the question is asked if the public wants governemnt involvement in health care, and a vast majority answer yes. The conclusion the report writers advocate is that this means that the public wants health care reform with a public insurance option. But the question is as relevant as asking people...do you want Medicare and Medicaid?...and having the answer come back "Yes". Both are government plans. To construe the answer as meaning that X% of the population want Obama health care reform is only achieved by an injection of report writer bias...and biased reporting is an area where poll data goes wrong.

A more relevant question would be: "Are you in favor of the bill that Congress is voting on now"...with a question also asked of "Have you read the legislation?"

The poll results would be interpreted quite differently if the report said that X% of Americans (or party affiliation) who have never read the legislation (and know who cashes in form it) are for or against said legislation.

Publish the polls, and leave out the biased interpretations that misconstrue the results. Ask more relevant questions to the bills being discussed.

[earlysaid]

I don't know about most people, but I know exactly who I voted for and I will know who it was to the day I die. I proudly voted for Barack Obama in the primary and the final election. America desperately needed a new direction away from every bumbling harmful incompetent and hateful thing done by the Bush administration. Republicans were totally incompetent blobs who did as they were told. Now republicans will only vote against what President Obama does to help the country. It doesn't take a genious to know that the GOP have become a pathetic group of people who have lost their soul and decency. They will never have the best interests of the America people in their thinking. They work for their uninformed and hateful base.

[schmidtyphi]

It's an interesting point -- people not "recalling" their vote correctly.

But, the numbers here are quite amazing: 48 to 25? What would explain that dramatic skew? The Kennedy assassination was traumatic and also 4 years after the fact. It's only been 6 months for Obama, and, I don't see any indication that his popularity has increased.

For me, in order for this article to be a meaningful explanation of the data, it would have to include an explanation of this incredible skew.

The article notes own people were reluctant to admit they voted for Bush, but enthusiastic to admit they voted for JFK. Why? And, how does Obama's administration reflect the latter and not the former?

If anything, my guess would be people are less likely to admit they voted for this guy. His approval numbers are actually significantly down since the election, for example.