E-Mailbag: Poll Position

By: Brian Montopoli
Topics: CBS News Issues

We got an email from Mark Blumenthal, the Mystery Pollster, concerning a point made by a commenter on his site about CBS News poll reporting standards. Blumenthal has been writing about a controversy over an American Medical Association-sponsored survey of young women about spring break which was covered by a number of news outlets, including the "Early Show." He pointed out that much of the reporting about the poll was flawed, in part because much of it relied on an AMA press release incorrectly identifying the respondents as a "random sample" of 17-34 year old women. (Mark has a lot more analysis here and here.)

The poll prompted one of his commenters, "JeanneB", to write the following:

More and more news outlets have hired ombudsmen in recent years. They also need to add someone to filter polls and ensure they're described accurately (if they get used at all)...

They have a responsibility to police themselves and set some kind of standard as to which polls will be included in their coverage and how they will be described. Please don't let them off the hook for swallowing whole any old press release disguised as a "poll".

Mark asked us to address the point and the issue.

I talked to Kathy Frankovic, CBS News Director of Survey, who emailed to me the updated standards for CBS News poll reporting. Here they are:

Polling is a complicated and often controversial method of research. Not everyone has the background to do it. CBS News does its own polling and reports on various outside polling efforts. Before any poll is reported, we must know who conducted it, when it was taken, the size of the sample and the margin of statistical error. Polling questions must be scrutinized, since slight variations in phrasing can lead to major differences in results. If all the above information is not available, we should be wary of reporting the poll. If there are any doubts about the validity, significance or interpretation of a poll, the CBS News director of surveys should be contacted. The CBS News Election and Survey Unit will maintain a list of acceptable survey practices and research organizations.

Whenever major poll results are reported on the air, we should include the pertinent facts about who conducted the poll, when it was taken and the margin of error. In the reporting the results of the poll, each question should be phrased accurately (verbatim is always best) immediately preceding the results.

In some cases, poll information may be shorthanded, such as, "In recent polls, the President has been getting high approval ratings from the American people." However, such assertions should only be made when the reporter can cite specific poll findings that support the claim. When the poll itself is the story, we should err on the side of providing too much, rather than not enough, information about its specific characteristics.

A word about the rapidly growing number of nonscientific polls: these are call-in polls, Internet and Web-site polls, and other surveys that inherently are not representative of the population. Such "pseudo-polls" should not be confused with scientific surveys and should never be reported as indicators of opinion.

The sections that are underlined represent new additions to the CBS News standards.

Brian Montopoli
Brian Montopoli is the senior political reporter at CBSNews.com.
Follow on Twitter »

3 Comments Have Your Say
Email Story Send to a Friend
Share This Tell Your Friends
Tweet This Tweet This
More Share It

Add a Comment


by tomgrey2 April 10, 2006 10:39 PM EDT: Polls are not real news, except the ones that count. Elections. And shopping purchases. The "news" that many people have one opinion or another seems pretty wimpy. As compared to the news that X, or an estimate of X, people were murdered in Darfur, or beaten up in Zimbabwe, or had acid thrown in their faces in Pakistan ...; Reply to this comment


by bob___k April 10, 2006 9:27 PM EDT: Replace, "If all the above information is not available, we should be wary of reporting the poll." with 'If all the above information is not available, we SHOULD NOT report the poll until it is all available.' "... the CBS News director of surveys should be contacted." Replace 'should be contacted.' with 'must be contacted for approval before running the item'. 'Should' does not require adherence. 'Must' does. "Whenever major poll results are reported on the air, ..." Replace 'major' with 'any'. Don't let someone subjectively decide what's major. "... each question should be phrased accurately (verbatim is always best) immediately preceding the results." Here 'should be phrased accurately' is very subjective, and does not require adherence. Verbatim is the only acceptable standard. If the question is excessively long the repondent may not have understood the question to be the same as now being paraphrased subjectively. "However, such assertions should only be made when the reporter can cite specific poll findings that support the claim." Here, 'can cite' should be changed to 'cites'. Transparency is better than trust. It seems your updated standards are rather loose. When were they updated? If these are the updated standards, how poor were the old ones?; Reply to this comment


by chubletst April 10, 2006 6:09 PM EDT: the day katie couric signs on, i am changing the channel. i left the today show after 25 years because she is tne story, not the news. bob schieffer has been wonderful, russ mitchell would have been great, john roberts should have had the job and probably for a lot less money. what you are paying her is insane. good luck and hello nbc, i do like brian williams.; Reply to this comment