Why Question Order Changes Poll Results

Kathy Frankovic: Change The Order Of The Questions In A Poll, You Change The Outcome

(CBS) 
To its credit, Gallup conducted a split-ballot experiment on the very next day. [I describe split-ballot experiments here]. Randomly-selected halves of the sample were asked the favorable question only one of the two ways. The original way of asking the question produced results that suggested there had not been a big change: how the questions were introduced had made all the difference. By August of 1998, “Thinking about Bill Clinton as a person” was much less likely to yield a positive assessment than “What is your overall opinion of ….?”

(By the way, there is a great review of the 1998 questions from all organizations in Jeffrey E. Cohen; Presidential Studies Quarterly, Vol. 29, 1999)

The tricky issues of what-follows-what, and how that affects the answers, aren’t simple. The safest way of minimizing any expected order effects is to rotate the order of the questions we ask about people and candidates. In fact, CBS News has been rotating question order when we ask voters whether they have a favorable or unfavorable view of the presidential candidates. We have used identical wording (“Is your opinion of [the candidate] favorable, not favorable, undecided, or haven't you heard enough about [the candidate] yet to have an opinion?”). The last choices (“undecided” and “haven’t heard enough”) are not universal among polling organizations. It’s something we uncovered back in 1984, when we realized that respondents often wanted to say they were “undecided” about someone whom they had, in fact, heard about.

In our latest CBS News/New York Times Poll, many registered voters do tell us they aren’t sure what to think about the remaining candidates. Nearly everyone has heard “enough” about Hillary Clinton and Barack Obama, but 12 percent still haven’t heard enough about John McCain. That probably reflects the more intense news coverage of the continuing Democratic battle. And just about one in five voters say they have heard enough but are not yet expressing an opinion. And it’s the critical independent voters who will almost certainly decide the November outcome - who are the ones most likely to be waiting to decide what they think about these candidates. And tracking them might not be as easy as simply ordering the questions!

[veteran72]

You know that the Bush/Cheney Crime Cartel has crunched the numbers and knows they''re headed for the Little Big Horn come November. Even with the MSM giving McBushcain a complete pass, he barely shows on the radar......that''s what they get for responding to
all reports of their crimes and incompetence with: "So"???

[joyous88]

All this is really indicative of is the simple fact

that in the United States the conservatives have

dumbed down education to the point that a high

school education is a waste of time.

Most people do not have the skill in statistics to

understand that they are in reality being manipulated

by the corporations.

You can not believe most of what you read from the

corporatist media, remember, One guy owns it all, and

he

is a greed driven conservative.

[it_oldtimer]

What an utterly nonsensical title for an article!

Without actually reading the article I wouldn''t have had a clue what the subject being presented in it was.

The total lack of basic literary skills shown by so many supposedly "professional" journalists these days has become truly appalling.

[cyberus-2009]

Not news at all .. thats how polls are manipulated.
Changing the order of questions is the easiest .. asking the same question using different words with the same meaning can alter the results.
Thats why there are so many pollsters out there .. every one of them are looking to get a result that will sell to someone.