On Petraeus and Casualty Numbers

The numbers game with regard to Iraqi civilian casualties is frankly a mind-bending experience.

UPDATE:  Added numbers for Iraq Body Count to the Charts

UPDATE #2:  After adding the IBC numbers, I noticed something strange that perhaps makes Ilan's analysis slightly less compelling.  See the conclusion below.

Based on queuing from a fine gentleman over at Intel-dump, I took a look at an analysis by Ilan Goldenberg over at Democracy Arsenal.  I recreated his chart, though our numbers are slightly different because they are derived not from actual data, but extraction from separate excel charts.  To that chart, however, I added various civilian casualty numbers as reported by others so that all (or at least most) of the known data can be examined together. Here's the chart, with further explanation and clarification below (click for a larger image):

Here's the line-by-line details of what the numbers are and who reported them:

• MNF (+wounded):  These numbers are derived from the latest quarterly MNF report (see the chart on page 20).  Like Ilan, I estimated the daily average for the month (the green portion) and then multiplied that number by the number of days in the month to arrive at a monthly figure.  Obviously there is some imprecision involved, but I imported the chart into photoshop and laid down a grid to improve the estimation somewhat. The most important difference between these numbers and the others is that they are casualties and not just deaths.
• Petreaus:  These are numbers derived from General Petraeus' briefing slides for Congress.  Like the MNF portion, I don't have the raw numbers, but got a decent estimation based on the slide.
• Icasualties:  These are the hard-data numbers as reported by icasualties.org and taken directly off their website.
• GOI: These are numbers reported by the Government of Iraq, as reported through various media sources, primarily the Associated Press.  The GOI does not publish them directly, so some searching was required to obtain all the months through press reporting.  In some instances, I found minor discrepancies in the numbers - in those cases, I used the higher number.
• UN:  Until January 2007, the UN calculated civilian deaths by using GOI and other information.  However, in January the GOI stopped providing the UN this info and so the UN discontinued the production of such estimates.  As a result, there is no UN data for 2007, and therefore it doesn't directly reflect on the surge, but I include it anyway as a comparison to the 2006 data.  I pulled the UN numbers from Spencer Ackerman at TPM (along with some GOI numbers).
• IBC: These are compilations of the raw numbers from the Iraq Body Count incident database.  The database has two numbers because of conflicting or unclear reporting - a minimum and a maximum.  The numbers I use in the chart are the maximum figures.  The IBC numbers include police and Iraq soldiers.  I did not want to take the time to edit them out by hand so I left them in.

I will leave it to the reader to make sense of the spaghetti soup except to point out two things:

1. First I find it interesting that the data beginning in April of this year is surprising consistent, especially when compared to other time-frames. Not only do the peaks and valleys match, but they are also relatively close in terms of real numbers and match very closely in terms of variation from month-to-month.  The divergent graph is, naturally, the MNF numbers which included wounded.



2. Secondly, the circled area in the chart at right identifies the discrepancy that Ilan noted in his post between the  MNF and Petraeus numbers.  As the chart shows, for December and January, the MNF numbers for killed and wounded civilians are LESS than the Petraeus numbers for killed alone - a seemingly impossible conclusion given that  both sets of numbers are ostensibly derived from the same source (specifically, the MNF-I SIGACT database).  It's also interesting that the Petreaus numbers for December 2006 are higher than all the other reported figures for that month. UPDATE: After adding the IBC and UN data, I noticed that their reported number of deaths for December, are, like Petraeus' numbers, greater than the MNF numbers for both killed and wounded.  ISTM that might indicate that the MNF numbers, at least for December, are flawed, but one must keep in mind that precision and accuracy on civilian casualties tough to quantify.

Given that we don't know how the various numbers in the MNF report and Gen. Petraeus' charts were derived, the only comment I'll make is that the discrepancy should be explained and that it would be helpful if MNF-I was more transparent with it's data and methodology.  There are a few explanations for the discrepancy beyond "cooking the books" but  the current political atmosphere, intense scrutiny, and, more importantly, very real policy implications suggest to me, at least, that an explanation should be provided as well as a greater degree of transparency - even at the cost of releasing some marginally classified data.  As it stands, and unless I am missing something (as I am definitely NOT a statistician), the discrepancy clouds the issue and certainly provides ammunition for those accusing Petraeus of "cooking the books" even though it is not definitive proof.

Comments

[Anon]

Just thought I would add to your questioning of the Petraeus numbers that the head of the GAO in testimony on their benchmark progress assessment continually stated that he was not comfortable with how Multi-National Force-Iraq was calculating its casualty numbers. Especially in the area of sectarian deaths.

[sheerahkahn]

Well, for me the bone of contention is that Petraeus is showing a decline in deaths, and the Icasualties shows an increase in deaths...which is a direct contradiction to Petraeus numbers...looks like near perfect X the way the two past each other.
/sigh
But I do agree with you on one issue...

"There are a few explanations for the discrepancy beyond "cooking the books" but the current political atmosphere, intense scrutiny, and, more importantly, very real policy implications suggest to me, at least, that an explanation should be provided as well as a greater degree of transparency - even at the cost of releasing some marginally classified data. As it stands, and unless I am missing something (as I am definitely NOT a statistician), the discrepancy clouds the issue and certainly provides ammunition for those accusing Petraeus of "cooking the books" even though it is not definitive proof."

And I think we both know that we'll have to wait beyond the current Administration's tenancy before the truth is outed.