Forbid Or Allow?

CBS' Kathy Frankovic: How A Poll Question Is Asked Can Sometimes Make A Difference

(CBS)  By Kathy Frankovic, CBS News director of surveys

---

When Iranian President Mahmoud Ahmadinejad spoke at Columbia University last week, crowds of protesters argued against that invitation. And when university President Lee Bollinger gave his introduction to that speech, he criticized Ahmadinejad.

I couldn’t find any polls conducted about the speech, and whether Americans (or New Yorkers) would have been willing to have Ahmadinejad speak at Columbia. But measuring a question like that may not be as easy as it seems. Questions about who has the right to speak have generated experiments that date back decades.

As far back as 1941, in fact. That’s when Donald Rugg published an article in Public Opinion Quarterly, reporting on a test of two questions about speeches against democracy. He did what’s known as a “split-ballot experiment.” One half of the sample, randomly selected, was asked: “Do you think the United States should allow public speeches against democracy?” The other: “Do you think the United States should forbid public speeches against democracy?” It’s fairly easy to divide a sample into random halves. Rugg could have had his interviewer alternate the question from respondent to respondent, or flip a coin to determine which question to ask. Nowadays, you can use the last digit of a phone number (odd vs. even), or just have a computer program randomly make the assignment.

Questions about allowing or forbidding speeches against democracy seem like mirror images of each other, but what turned Rugg’s experiment in question wording into a journal article was that they were not. Seventy-five percent of Americans would “not allow” speeches against democracy, but only 54 percent would “forbid” them. The experiment became part of survey lore. Howard Schuman and Stanley Presser reported their 1970’s experiments with those two words in their 1981 book, Questions and Answers in Attitude Surveys. Hans Hippler and Norbert Schwartz tested the forbid-allow issue in 1982 in West Germany, and Bregje Holleman did so later in the Netherlands. Philip Gendall and Janet Hoek report finding forbid-allow differences in New Zealand.

In New Zealand in 1989, 13 percent agreed that “the law should allow public speeches which promote racism.” When the statement was turned around, so was the answer. Thirty-one percent disagreed with the statement that “the law should forbid public speeches which promote racism.”

That same year, in the U.S. General Social Survey, 19 percent said “yes” when asked “Do you think the law should forbid marriages between blacks and whites?” But 32 percent said “no” when asked “Do you think the law should allow marriages between blacks and whites?”

Some of these differences are not just because people are forced to choose between yes and no. Similar results have been found on some questions that give respondents a scale of possible answers.

What is going on? One easy answer is that although it may appear that forbid and allow are two sides of the same coin, they aren’t. “Forbid” is a strong negative; while “allow” might imply not just permission, but perhaps even approval. Another is that people without strong opinions about a subject will not endorse either option, causing the difference.

The good news is that these double-digit differences are the exception, and not the rule. And that the practice of conducting split-ballot experiments continues.

CBS News and The New York Times routinely add these sorts of experiments to our questionnaires. When candidates use different versions of their names (like “Hillary” and “Hillary Rodham” Clinton or “Rudolph” and “Rudy” Giuliani), we want to find out whether the name makes a difference. Our good news is that when we have conducted such experiments recently, it hasn’t.

And things sometimes change. For example, we have asked Americans to tell us whether the war in Iraq has been worth the costs, defining the outcome as “the result of the war” to one randomly-selected half of the sample and “removing Saddam Hussein from power” to the other half. Earlier in the war, those different wordings resulted in different answers, with more Americans describing the war as worth the cost when we asked about “removing Saddam” than when we asked about “the result of the war” in general. But in more recent polls, there have been small or no significant differences in these questions.

So that 1940 experiment about “speeches against democracy” might not get the same results today as it did then. In a 1989 General Social Survey experiment about a speech by an “admitted communist,” 34 percent said they would “forbid” such a speech, while 35 percent said they would “not allow” it - no significant difference at all.

Learning what we can about the impact of question wording makes polling both more interesting and more challenging. And had we polled on Ahmadinejad’s speech last week, we might very well have found America’s answer - however we asked the question!

[cbs_oliver]

The best way to find out about people''s views is to ask them about their own actions or willingness to take actions - taking into account that they may not tell the truth. That bypasses rationalizations and gets down to the nitty gritty.

It is important though not to extrapolate from the question asked to pridict the likely answers to other apparantly similar questions. It is my experience that people who call themselves conservative, for example, use tribal and situational valuations rather than global and priciple based valuations - despite their claims to the contrary.

[cbs_oliver]

I agree with your general point rational_1.

The abortion issue is also interesting because most of those who say they want to stop abortions generally push to put the doctors in jail but not the mothers or others who order the abortion. The rhetoric they use does not fit the bills that they propose.

[rational_1]

I''ve thought for a while how people use different terminology depending on what side of an issue they are on, and the choice of terminology will obviously influence the poll results. The abortion argument is an excellent example of this. The Pro-life movement uses the positive term "pro-life" while the other side uses a negative, "anti-choice", to describe the pro-lifers. NARAL likes to describe itself as "pro-choice" rather than "pro-death", which would be a much harder sell. So, depending on which of these terms comes up in polls influences the polls. I''ll bet this applies to a lot of polls, which is why I generally ignore them.

[cbs_oliver]

A good article by Kathy Frankovic once again.

Some thoughts: I wonder how the results might change if the words "ought" and "should" were alternated as well as %u201Callow%u201D and %u201Cforbid%u201D. Ought is a stronger word.

Another phenomena is when a poll question is phrased in such a way as to impose (push) an evaluation scheme on the responders.

Some polls have asked "Was the war in Iraq worth it?" The responder is asked to calculate the value of the war in an abstract currency of their own choosing. Those who accept this request have accepted a utility valuation scheme for actions.

A poll could ask "Was the war in Iraq the right thing to do?" This is a completely different kind of question. Now the valuation is between right and wrong. Numbers are not necessarilly the issue.

[hideronpaul]

*** STOP THE WAR & END CORPORATE CORRUPTION ***

Who Is Paying For Your Vote To Keep Control Of You And To Keep This War Going? Remember The Bankers Hold The Stock In Companies That Are Getting Billions In War Contracts!

Obama''s Contributers: Bankers / Special interest!
http://tinyurl.com/2nd4f8

Hillary Clinton''s Contributers Lawyers / RealEstate:
http://tinyurl.com/2ontpq

Rudy GIULIANI Contributors: Bankers / Hedge Funds!
http://tinyurl.com/2m2c4n

Ron Pauls Money Is From the People! We have someone who is NOT Paid Off By Special Interest or Bankers Money!!!
http://tinyurl.com/2q8vr3

If you ever want this War to End Vote Ron Paul 2008, No One Else Is Going To End It!
http://www.ronpaul2008.com/issues

Ron Paul Will:

** Stop Iraq War Immediately
** Eliminate IRS
** Eliminate Federal Reserve
** Eliminate Government Wasteful Spending
** Restore America''s Work Force & Values
** Restore America''s Freedom!

RON PAUL RAISES $5.1 MILLION 3RD Q!
FROM THE PEOPLE! VOTE RON PAUL 2008!

[kansas1946]

How about the attack on Coulter?
**************************
You aren''t serious...are you?? Coulter??

Now on to attack, versus allow or forbit. I think folks are a bit confused about free speech. Free speech means you can say what you want against the govenment and that they can not prosecute you for it, put you in prison, etc.

For instance, I can say that President Bush is a brain dead idiot, and the FBI is not going to show up at my door, kick it in, and haul me off, for saying it.
Now if you like Bush, you can verbally attack me. You can call me an idiot, a traitor, or anything you like. I am not protected from your speech, or you from mine.
But both of our speech is protected FROM the government.

Ann Coulter can say anything she likes, and we, in turn can say anything we like about her, and it has nothing to do with free speech.

[xlib]

It would have been nice if an article like this was run after the Minutemen were attacked by the free speech loving libs at Columbia. How about the attack on Coulter? The list is long and the msm has no problem ignoring or condoning such actions.
As for the brain twisting statement "how a poll question is asked makes a difference", that deserves a big DUHHHHH.