Polling: Who's On First?

By Kathleen Frankovic, Sarah Dutton and Jennifer De Pinto of the CBS News Election and Survey Unit.

Those who follow the polls may be wondering how to make sense of all the information they contain. That's especially true for the "horse race" -- the polling question that asks participants which candidate they would vote for if the election were held today.

Right after the Republican Convention, polls released within days of one another ranged from a double-digit lead for President Bush to a tied race. CBS News' most recent poll had Mr. Bush leading John Kerry by eight percentage points among registered voters (with likely voters not much different).

Why are these polls, conducted by reputable pollsters with many years experience in the polling business, showing such different results? There are a number of reasons why the polls differ.

The order in which the poll questions are asked can make a difference in the results. Most pre-election pollsters try to ask the vote questions early in the questionnaire, before they have introduced other topics and issues. Bringing up current events or issues could "remind" voters about things that might influence their vote.

For example, a survey conducted by the Pew Research Center for the People and the Press (which regularly polls on current events, politics and the media) first asks respondents how much attention they have been paying to a number of recent news topics before asking whether people are registered or how they will vote. While in theory this should not affect how people vote, it may focus attention on certain issues – and mentioning Iraq, the economy, and the ad campaign by the "Swift Boat Veterans for Truth" can affect the responses of less committed voters.

Some polls report results among registered voters, and some among likely voters. These are different groups (likely voters are a subset of registered voters). Before comparing polls, it is important to determine which group the results refer to. It is also helpful to know how "likely voters" are determined.

Each organization has its own formula for identifying likely voters. Each polling organization has developed its own method for doing so, and these techniques vary. They usually involve measures of past voting behavior, reported intention to vote, and interest in or attention to the campaign.

Some organizations ask only one question. On the other hand, The Gallup Organization uses a series of questions that were first developed to be used in the last weeks of a campaign. Applying that series for the entire year before the election may have some unintended consequences and as we've seen, the Gallup Polls of likely voters have shown greater swings than many other polls.

Additionally, some polling organizations limit the number of voters included in their "likely voter" group, not letting the number exceed their expectations for what turnout will be. That can exclude some voters who would fit another polling organization's definition of likely voters.

The dates the interviewing was conducted could affect the poll results. For example, a few polls were either conducted or interviewing was begun before the end of the Republican convention, and before the president had given his speech. Other pollsters waited until a few days after the convention to conduct their polls. The first presidential debate is scheduled for Thursday, and polls conducted before the debate may be very different from those conducted afterwards.

Do polls assume a certain party distribution on Election Day? Some pollsters assume that party identification is relatively stable, and that they should weight the sample to reflect the answers that people give to a question about their party identification (Republican, Democrat, or Independent). But many states don't register voters by party, and many poll respondents may be answering a question about which party they identify with based on what they've just heard during the survey – and may very well think differently when they've gone to the polls and just cast a ballot (for many, the most partisan action they will ever take).

When comparing poll results, it is important to compare apples to apples. Polls that report the results among different groups of voters (registered versus likely), or that ask questions in a different order, or that make specific assumptions about Election Day voters, simply are not comparable.

In fact, right now the polls are fairly consistent. Taking a look at the most recent major pre-election polls among registered voters shows similar results. All of these polls – at least for now -- report a lead for Mr. Bush over John Kerry.

RECENT POLLS (among registered voters)


	Date	Bush	Kerry	Nader	Bush Lead

CNN/USA Today/Gallup	9/24-26/04	53%	42%	3%	+11pts

ABC News	9/23-26/04	51%	44%	2%	+7pts

Time	9/21-23/04	47%	43%	6%	+4pts

AP/Ipsos	9/20-22/04	51%	42%	2%	+9pts

CBS News	9/20-22/04	49%	41%	2%	+8pts

NBC/WSJ	9/17-19/04	48%	45%	2%	+3pts

Gallup	9/13-15/04	50%	42%	4%	+8pts

Newsweek	9/9-10/04	49%	43%	2%	+6pts

Pew	9/8-14/04	49%	43%	1%	+6pts

For example, the latest NBC/Wall Street Journal poll shows Mr. Bush with a 3-point lead over Kerry. The latest CBS News poll gives the President an 8-point lead. Why are they different?

The dates over which the two polls were conducted are fairly close. Numbers from both polls are among registered voters. But the NBC/Wall Street Journal Poll asks quite a few questions that CBS News does not ask before asking respondents for whom they would vote.

CBS News asks respondents whether they are registered to vote, how likely they are to vote in November, and how much attention they are paying to the election before asking the vote question.

The NBC/Wall Street Journal Poll asks registered to vote, interest in the elections, whether the country is headed in the right direction, approval of the job Mr. Bush is doing (as president, handling the economy, handling foreign policy and dealing with terrorism) and whether Mr. Bush deserves to be re-elected before asking for whom they would vote.

Each organization has its reasons for structuring questionnaires as it does. Nevertheless, differences in question order could well account for disparities in the horse race results.

More from CBS News