Sticking To Standards Key With Polls

By Kathy Frankovic, CBS News director of surveys

Any poll consumer needs to be able to distinguish between good polls and bad polls. But how do we teach people to do that? This isn't just a question for pollsters, but for journalists, too. Sometimes even governments get involved. There is a lot of public skepticism about polls - many people don't believe that well-done polls are any better than polls that aren't representative of anything. And when asked, the same people who give their opinions to pollsters also say they don't believe in poll results!

In 2000, only 32 percent of Americans told Fox News they believed polls were right most of the time (59 percent said they were right only some of the time). Americans really haven't accepted the modern scientific polling process - though probability sampling has been the standard for more than sixty years. Just 21 percent of Americans interviewed in a 2001 Gallup Poll said they thought "a sample of 1,500 or 2,000 people can accurately reflect the views of the nation's population." Seventy-five percent said this was "not possible with so few people." And only 48 percent told CBS News in 1998 that they believed polls during elections when they showed one candidate ahead.

There is also not a lot of love lost between public and pollsters. Last year, 54 percent of Americans said they could not generally trust pollsters to tell the truth.

Still, Americans want politicians to pay attention to public opinion, and sometimes that means relying on the same polls that people would otherwise criticize. Depending on the time and the question, the percentage of Americans who describe polls as generally a good thing has ranged from 68 percent (CBS News, 1998, during the height of the Monica Lewinsky scandals) to 89 percent (Gallup, 1996). More than 80 percent told the Kaiser Foundation that conducting public opinion polls was a good way for the government to learn about public opinion. Ninety-one percent told CBS News in 2003 that the government should take the views of the people "as expressed in polls" into account when making decisions about the war in Iraq. And in September 2005, Gallup found that 61 percent of Americans thought the country would be better off if leaders "followed the views of public opinion polls more closely."

Most news organizations, including CBS News, have standards for reporting polls. This is an excerpt from the CBS News Standards:

"Before any poll is reported, we must know who conducted it, when it was taken, the size of the sample and the margin of statistical error. Polling questions must be scrutinized, since slight variations in phrasing can lead to major differences in results. If all the above information is not available, we should be wary of reporting the poll. … Whenever major poll results are reported on the air, we should include the pertinent facts about who conducted the poll, when it was taken and the margin of error. In reporting the results of the poll, each question should be phrased accurately (verbatim is always best) immediately preceding the results."

The CBS Standards go on: "In some cases, poll information may be shorthanded, such as, "In recent polls, the president has been getting high approval ratings from the American people." However, such assertions should only be made when the reporter can cite specific poll findings that support the claim. When the poll itself is the story, we should err on the side of providing too much, rather than not enough, information about its specific characteristics. "

The CBS News Standards also prohibit the use of pseudo-polls, especially self-selected samples like telephone call-ins and Web site polls.

Our rules are similar to the disclosure requirements of professional American and international survey research associations. But there is no law in the United States that requires a news organization to provide any such information alongside a poll result.

The First Amendment really does matter here. In some other countries - like Greece - there actually are laws about what makes a good poll. At least 1,000 people must be interviewed in a pre-election poll, according to Greek law, and the questionnaire and the data must be made public.

Russia - yes, Russia - wrote reporting requirements for pre-election polls into its 1995 election law. When reporting pre-election polls, news organizations must report who conducted the poll, when, how, and in what portion of the country it was conducted, how many people were interviewed, the question wording, the statistical margin of error and who paid for the poll.

It's a good list, and while no one in America would want to have them legislated into law, we could do worse than follow the Russian example when reporting polls!

By the way, a few weeks ago I wrote about the Greek election laws that forbid reporting pre-election polls in the two weeks before the September 16 election. A few days afterwards, the man who proposed that law, Theodoros Roussopoulos, the government minister responsible for the press and information, announced that the two-week restriction on publication would be abolished and replaced with only a 24-hour limit! The former journalist realized that restricting information might not be such a good idea.
By Kathy Frankovic