With Polls, Trust But Verify

Ballot box generic us american flag vote poll capitol building polling positions frankovic CBS/AP

By Kathy Frankovic, CBS News' director of surveys.



Polls say many things — but sometimes they don't — really.

I've been conducting public opinion polls for CBS News since 1977 — 30 years. That's a long time by anyone's standards, but examples of misread and misinterpreted polls don't only come from 30 years ago. Even today, they take place far too often.

One of the biggest problems in reading public opinion polls is placing too much confidence in them — taking numbers from a survey and reading them as a precise measurement of where the public stands. Some polls even insist on providing the results to a tenth of a percentage point, while admitting their results could vary by as much as three or four full percentage points just because of a sampling error.

Too many pundits (and pollsters) spend time trying to explain a one or two point change in ratings based on current events when there is a greater likelihood that the change really is due to small fluctuations in the sample — the margin of error.

Frankly, there is no practical — or political — difference between a presidential approval rating of 34 percent (the latest USA Today/Gallup poll), 33 percent (the latest Gallup poll) and 32 percent (the latest CBS News/New York Times poll). In every case, only about one in three Americans approve of the way the president is handling his job. It's bad news for the incumbent any way you look at it, with similar political impact.

There are other problematic and misleading interpretations. National pre-primary polls taken in the year before an election may say nothing about the outcome of even the first primary.

In March 1991, Bill Clinton wasn't even on the list of candidates proposed to the public in an NBC News/Wall Street Journal poll as possible Democratic nominees. New York's Gov. Mario Cuomo was Democrats' top choice. Of course, that was a different era. As late as that June, only Massachusetts Sen. Paul Tsongas had actually officially declared himself a Democratic candidate.

And in that same time period — one and a half years before the 1992 presidential election, then-President George H.W. Bush led the generic "Democrat" by nearly two to one in 1992 horserace questions. After the successful completion of the 1991 Persian Gulf War, the president's approval ratings were high. But by 1992, they were not. Pat Buchanan challenged Bush in the Republican primaries, and Bill Clinton easily won in November.

Current polls (including the CBS News Poll) suggest that a generic Democratic candidate would easily defeat a Republican candidate in the 2008 presidential election. But that may or may not happen. Polls taken now can't be used as a prediction of what will happen after more than a year.

All the poll reviews and all the academic studies of pre-election polls tell us that polls are very good predictors close to an election. Right now, they only reflect the public's current knowledge and perceptions, and the current state of the country. But things can change.

We've all seen poll reports of questions that ask Americans what they think other Americans think. Those are great for telling us whether people assume they themselves are in the majority of the minority on an issue.

For example, before an election we routinely ask voters not just who they plan to vote for, but who they think will win the election. Usually, one candidate's supporters are a lot more hopeful than the other candidate's, and that can be an indication of voter enthusiasm. In the days immediately before the 2004 election, 84 percent of Bush voters were confident that their candidate would be reelected, but only two-thirds of Kerry voters believed Kerry would win.

But don't assume these questions are always good predictions, even though Bush beat Kerry in 2004. The question was about each individual's optimism or pessimism about winning in 2004, but it also reflected media coverage and what "experts" are telling attentive voters. We sometimes ask people to tell us why an event happened, or what is in other people's minds, but their answers don't give us a definitive answer to those questions. The public isn't an "expert" on psychology, or on the motivations of politicians. They are experts on themselves!

Some poll questions tell us mainly what pollsters are thinking. The Roper Center at the University of Connecticut maintains a searchable database of nearly all public opinion poll questions, including their results.

My all-time favorite finding says a great deal about both the increasing numbers of polls and how professional pollsters judge what's important. There are nearly three times as many questions that include the name "Monica Lewinsky" than include the word "Watergate." Until just last year, Monica Lewinsky was the most frequent woman's name in public opinion poll questions, with more than 1,500 questions that included her name.

Last year, there was a new woman front-runner, and her name was Hillary Clinton.
By Kathy Frankovic
  • David Miller

Comments