News Outlets Sweat Over Exit Poll Accuracy

Media outlets are preparing for the possibility that their Election Day surveys could be skewed because of overstated support for Barack Obama, largely because of the enthusiasm of his supporters.

While exit polling is a notoriously inexact science—early exit poll results suggested John Kerry would be elected president in 2004—the introduction of several new variables, ranging from the zeal of Obama’s supporters to his racial background to widespread early voting, is causing concerns among those who charged with conducting the surveys and the networks that will be reporting them.

“It’s in some ways the flip side of non-cooperation,” said one pollster involved in preparations, who spoke on the condition of anonymity, “It’s over-cooperation by certain people.”

Unlike a traditional poll that asks voters who they intend to support, exit polls are taken outside the polling place immediately after voters have cast their ballots. The interviewing begins when the polls open in the morning and lasts throughout the day until shortly before the polls close.

The exit polls are conducted by the National Election Pool (NEP), a consortium of ABC News, Associated Press, CBS News, CNN, Fox News and NBC News formed in 2003.

In theory, exit polls should match election results. But for all the care that goes into conducting accurate exit polls, errant results aren’t completely uncommon. Respected polling analyst Mark Blumenthal found that during the Democratic primaries this year, preliminary exit polls overestimated Obama's strength in 18 of 20 states, by an average error of 7 percentage points, based on leaked early results.

The reason? Obama’s supporters were younger, better educated and often more enthusiastic than Hillary Clinton's, meaning they were more likely to participate in exit polls.

Insurgent candidate Pat Buchanan’s support also was overstated in the 1992 New Hampshire Republican primary, a phenomenon attributed to the greater willingness of his impassioned supporters to participate in exit surveys.

More recently, in 2004, exit poll data that began circulating early in the afternoon led to short-lived Democratic elation and deep Republican anxiety. By evening, some of President George W. Bush’s key strategists were frantic, emailing reporters at polling organizations to better understand the gap between what they were finding on their own and what the leaked exit polls indicated.

As it turned out, preliminary exit polls overstated women’s turnout that year. This “programming error,” which affects the statistical method that pollsters use to match surveys to the electorate’s composition, was discovered by the third wave of exit polling.

By then, though, the premature polls had already been leaked online.

While the exit poll questionnaire—what’s asked and how—is not yet decided upon this year, as part of their Election Day preparations many news organizations are conducting rehearsals next week to prepare for possible hiccups.

“We know there are concerns about exit polls,” said Kathy Frankovic, who directs CBS News polling unit. “Our goal is to do two things on election night: explain what’s going on and why, and to accurately project outcomes.”

One important difference between 2008 and 2004 is that the early poll results are less likely to leak. Just as in 2006, the NEP has isolated a room in New York City where analysts are closed off to communication.

Accurate sampling, however, is an even greater concern. In 2004, the over-sampling of women—who are more likely to be Democrats—was not the only error. Democrats, in general, turned out to be more likely to participate in the exit poll interviews than Republicans.

It also surfaced that younger interviewers, who hand out the paper questionnaire to voters, were less likely to get a response from lder voters. Older voters are more likely Republican.

In the end, Kerry’s vote was overstated in 26 states. The same was true for Bush in four states, according to a detailed post election analysis by Edison Media Research and Mitofsky International, who administer the exit poll for the NEP.

To correct their methodology, Joe Lenski, executive vice-president of Edison, said the NEP has increased the average age of those conducting interviews. In 2004, that age was 34 years old. Today, the average is 42.

In addition, the NEP has successfully won the right in several state courts to allow their interviewers to gain closer physical access to polling places. This allows pollsters to measure a more accurate sample. There has also been increased training of interviewers to maintain random sampling—for example, cautioning interviewers to not habitually approach the next approaching voter if one voter declines.

“The lessons of ’04 that have been applied to exit polls since then are that interviewers need to have better training, that you need to be cautious in looking at the results,” Frankovic said. “And the world is going to be grabbing at the information as soon as it’s out there and you can’t control what the world says about it.”

But the NEP will control when the world has access to the data. Each news organization sponsoring the massive 50-state survey will have three representatives in the closed-door analysis room. Not until 5 p.m., when at least two waves of polling are completed in most states, will the media sponsors have wider access to digest the data.

And, unlike with a standard phone poll, the staff handing out the forms can track the race, gender, and estimated age of those who decline to participate. That could matter since pollsters have noted that older or less educated voters are often less likely to participate and those voters are more likely to have latent racial sentiments.

In general these errors, when tracked in real time, can be corrected for by pollsters before the results are reported.

Pollsters interviewed were confident that latent racism would not skew poll findings, such as if a white respondent votes for McCain but writes Obama on the questionnaire.

“I don’t think there is a reason to expect there will be an effect of racism, mostly because of the secret ballot,” said the pollster involved in preparations.

If anything, the impact of a massive increase in early voting is actually more of a concern than latent racism. The pollster involved termed it “the issue that is obviously on the minds on of exit pollsters.”

The NEP will be increasingly relying on phone polls conducted prior to Election Day to compensate for states with large portions of early voters--a technique that proved successful most recently in the Democratic primaries.