Polling Continues To Raise A Weighting Issue
It seems that every time CBS News conducts a poll we get questions. When CBS News conducts a poll that shows President Bush’s approval rating at an all-time low of 34%, the questions become a whole lot more skeptical. We’ve looked at this issue from a couple different angles in the past and we’ll probably go through it many more times.
Since the questions we receive almost always revolve around the issue of weighting – how many Democrats, Republicans, Independents, etc. are included in the polling sample – we’ll take you through CBS’ methodology once again. Here’s how the CBS polling unit describes it:
Update: Mystery Pollster recalculates the CBS weighting with same results.
Update: The Anchoress has compiled a roundup of blog reaction to the poll.
Since the questions we receive almost always revolve around the issue of weighting – how many Democrats, Republicans, Independents, etc. are included in the polling sample – we’ll take you through CBS’ methodology once again. Here’s how the CBS polling unit describes it:
At the end of our surveys, we find sometimes that we have questioned too many people from one group or another. Older people, for example, tend to be at home to answer the phone more than younger people, so there is often a greater percentage of older people in our surveys than exists in the American public.Other pollsters use different methods of weighting than the U.S. Census breakdowns, but this is commonly accepted method of polling. There are plenty of complex and debated issues within the polling community about their craft, which is sometimes described as part science, part art. If you want to really delve into the issues, check out Mystery Pollster for a fantastic primer and in-depth analysis.
When that happens, we take great pains to adjust our data so that I accurately reflects the whole population. That process is called “weighting.” We make sure that our final figures match U.S. Census Bureau breakdowns on age, sex, race, education, and region of the country. We also “weight” to adjust for the fact that people who share a phone with others have less chance to be contacted than people who live alone and have their own phones, and that households with more than one telephone number have more chances to be called than households with only one phone number.
So when we add up all the answers to our questions, we know that no one’s opinion counts for more than it should. When you see one of our poll results on TV or in the newspaper, you know that it does not show the opinions of only one or two groups of Americans.
Update: Mystery Pollster recalculates the CBS weighting with same results.
Update: The Anchoress has compiled a roundup of blog reaction to the poll.