Iraq Casualty Study: Cutting Through The Hype

(AP Photo/Hadi Mizban)

The Washington Institute for Near East Policy – which counts members of both sides of the political spectrum among its supporters (on its Web site, both Al Gore and Charles Krauthammer provide quotes praising it as a respected source of information) – has released a "critical examination" of the study. Given the amount of attention that has been paid to the Lancet study – and the spin that both sides of the aisle have put on its validity – it's well worth reading the Washington Institute's analysis. "The [Lancet] paper should definitely not be dismissed out of hand," write authors Jeffrey and Loring White, "however, it may suffer from some methodological and analytical weaknesses.(These 'scientific' concerns could be readily addressed with public access to the sample data.) In addition, it seems to display some bias in its interpretation of data, especially in its attribution of responsibility for the violent deaths in Iraq. It also includes several potentially biased and controversial statements about the coalition."

The authors examine some of these concerns in detail. Specifically, how the study authors arrived at their conclusions – and, notably, how commentary about the study has overlooked some specifics. From the Whites' analysis:

"Much of the commentary has skipped over the fact that the study presents a range of numbers (from approximately 393,000 to 943,000 deaths) rather than a single figure. Nevertheless, even the lower end of this range is substantially higher than any other estimate, as the study acknowledges. These figures are based on household survey data, which provided a sample (1,849 households) projected to the national level. Although the survey appears to have been conducted according to standard practices, some results do not seem plausible. For example, the numbers projected by the study would require the violent death of approximately 500 Iraqis per day for 1,200 days. Furthermore, given the study's estimates of violent deaths concentrated in a few provinces, daily death rates in these areas would have to be substantially larger than 100 per day for prolonged periods of time.

Nothing we know how about the conflict in Iraq supports this rate of violent death. There may have been a few periods when that rate was achieved, or even surpassed (e.g., in November 2004 when the conflict with the Sunni insurgents reached a peak in Falluja and Mosul), but such intense episodes of combat have been infrequent. Killing people in large numbers is not easy, short of employing weapons of mass destruction, and in the media fish bowl of Iraq things cannot stay hidden for long, as exemplified by the stories of atrocities committed by U.S. personnel. This is not to say, however, that other lower estimates are any better—many appear to be markedly undercounting fatalities.

UPDATE: Reader and Huffington Post contributor Maia Szalavitz writes in to direct us to two stories at Stats.org that examine the criticisms of the Lancet's methodology. You can read them here and here. Szalavitz shares her own thoughts on the matter at Huffington Post, arguing that the "methods used to estimate the death toll are widely accepted in the scientific community as valid." She also criticizes the media's coverage of the study as "curiously dry. Not one of the stories I read was humanized by the story of a particular death or incident-- all focused on the political debate and/or the supposed methodological problems, even as the actual experts on this kind of data didn't find them."