Early Polls Often Unreliable
By Kathy Frankovic, CBS News director of surveys
This year, voters are paying attention to the election campaign much earlier than usual — 69 percent in the most recent CBS News/New York Times Poll say they are paying at least some attention now. But does that mean that early horserace polls tell us what will happen next year?
Recent polling provides a good example of why we should be cautious — and maybe even a little skeptical — when reading poll measurements of candidate preference this early in a presidential campaign. About two weeks ago USA Today ran a front page story about possible changes in the landscape of the campaign for the Democratic nomination. The USA Today/Gallup poll showed the race for the Democratic nomination effectively tied — with only one point separating Illinois Sen. Barack Obama and New York Sen. Hillary Clinton — within the poll's margin of error. Clinton had been leading in all other national polls — and by a significant margin — so this poll appeared to mark a change in the race.
But this week, in the next USA Today/Gallup poll, results went back to their earlier patterns: Clinton maintained the double-digit lead over Obama she has held in nearly every other poll measurement. As reported in USA Today on Monday, Frank Newport, editor in chief of the Gallup Poll, said the previous poll "either picked up a short-term change or … was a function of unusual sampling, which happened to pick up Democrats who were more pro-Obama than the underlying population."
If even one poll can change so much because of no obvious campaign event, can we compare polls from different organizations? Frequently, we can't. This early in a campaign, different pollsters ask preference questions of different groups of people. For example, some polls ask their primary horserace questions to those who identify or "lean" toward the party being asked about. Respondents first answer a question about their general party identification (one version of that is: "Do you think of yourself as a Democrat, a Republican or an independent?") and those who say they are independent then are asked if they "lean" more toward the Democratic Party or to the Republican Party.
Other polls, including the CBS News/New York Times polls, ask primary horserace questions only to people who report they are registered to vote (about 80-85 percent of all adults interviewed). In addition, those registered voters are asked whether they are likely to vote in a Republican or Democratic 2008 primary or caucus.
Those differences make it almost impossible to compare results between polls. About three-quarters of the CBS News Democratic primary voters call themselves Democrats when they are asked their party identification. Fewer than two-thirds in Gallup's most recent Democratic preference poll were Democratic identifiers.
So what's going on in early campaign polls? Potential voters often choose candidates they are familiar with. Many announced candidates are simply unknown quantities. Even after his years in the Senate and a previous presidential run, 55 percent of Americans interviewed in an April Gallup poll still could not say whether they had a favorable or unfavorable opinion of Delaware Democratic Sen. Joe Biden. The earliest polls say more about name recognition than likely votes.
New candidates — such as Sen. Obama — often gain support in early polls when they enter the race. Voters don't yet know a lot about them, and what they know comes from the generally positive coverage, as well as the new entrant's effective status as "none of the above." For example, in January, more than half of registered voters who said they would vote in a Democratic primary/caucus had no opinion one way or the other about Obama. But 40 percent were favorable, and only 3 percent were not.
And then of course, there are the phantom candidates, not yet in the race, whose appeal says more about the candidates currently in the race than about how well the non-candidate will fare if he or she does join in. Former Tennessee Sen. Fred Thompson fits this category.
So was there real change in the Democratic horserace in the last few weeks? Was Obama first up and then down? Probably not. These early polls tell us about voter feelings, but they often have not been particularly good predictors of what will happen in an election. After all, four years ago, in June 2003, the Democratic leader in the Gallup Poll was Connecticut Sen. Joe Lieberman, Al Gore's running mate in the 2000 election. Missouri Congressman Dick Gephardt was running second. Whatever happened to the Lieberman-Gephardt ticket?
By Kathy Frankovic