let's put all our faith in an incomplete database

Even Jonathan Steele is perfectly happy with IBC.

The [Chilcot] inquiry closed its public hearings last month after seeing 140 witnesses but none dealt specifically with civilian casualties, which the IBC calculates as between 97,000 and 106,000.

Other groups have produced much higher estimates but the IBC, a UK-based organisation set up by volunteers to track deaths in Iraq, prides itself on working carefully with data drawn from cross-checked media reports, hospital, morgue, NGOs and official figures.

Gosh that sounds scientific. In contrast to all those 'other groups'. Careful working, data, lots of cross-checking, different sources...

Dear Jonathan Steele

Just over 2 years ago you wrote a detailed account of the different 'estimates' of Iraqi dead, and you even quoted the chief scientific advisor to the MOD, who described the Lancet study as 'close to best practice'. You may also be aware of a recent report put out by Landmine Action – A State of Ignorance – which reveals a number of other comments about the Lancet study, from sources inside the civil service and inside government. Here are a couple of quotes:

1. Part of inter-ministerial email correspondence leading the 17 November 2004 House of Commons and Lords statements about the Lancet study:

'I’m still worried about where we may be heading. Obviously if the estimate of 100,000 is wrong, we must make that clear. But for every flaw identified, there is testament to the study’s sound methodology. The Economist quoted Scott Zeger, head of department of biostatistics at John Hopkins that the clustered sampling is the rule in public health studies. Death by epidemic also varies by location. If this is how these people usually calculate the effects of epidemics, we need to be careful about criticising it, especially when we have made no attempt of our own to make an estimate – a very major weakness.'

2. A 'restricted' letter from ministry's chief economist, Nov. 2004:

"It might also be possible, as Gerald Russell has suggested, to try and validate the study’s preinvasion estimate of mortality by checking it against unpublished MoH health figures. But there is (a) no certainty at this stage that this kind of work would invalidate the Lancet findings, or (b) any guarantee that if it does produce a difference answer, that the rejection of the Lancet findings would be conclusive."

It appears from many of the FOI requests which appear in this report that those working for the government were far from confident that the Lancet study could easily be dismissed. Indeed, the general picture is that they were likely to be reliable, but needed to be rejected.

Are you yourself confident that the IBC's figures are close enough to reality to make them quotable in every case of discussing deaths in Iraq as a result of the war? If they are likely to be out by a figure of 5 or 10, or if it is even possible that they are out by factors such as these, are you happy for the figure in the public mind to be that of IBC?

I am aware that your article was about IBC, and you therefore had to quote their figures; and you did also mention 'other studies'. But you must be aware that the IBC figure is the one in everyone's mind, and that every instance of mentioning it, and of failing to give other studies at least an equal profile, only reinforces the perception that this is the authoritative figure.

I would be interested to know your comments.

Thank you
[*]

A reply...

Dear [*],

Thanks for your emails. The piece I wrote the other day was about an important critique of the inadequacies of the Chilcot inquiry. The IBC chose to make one. If the authors of the Lancet material or the ORB chose to make their own critique, we would probably cover it.

My article was obviously not, nor intended to be, an analysis of the various calculations of Iraqi casualties.

Separately, however, I would say that the IBC is a database, not a survey and extrapolation.

I take it seriously as a starting-point for some sort of approximation to the minimum number of deaths. The IBC itself admits that it undercounts them.

Best wishes,

Jonathan Steele

Approximation to the minimum?? Wouldn't it be better to approximate to the reality? Or at least to say how far the 'approximate minimum' is from the most likely reality.

Me again:

Dear Jonathan
Many thanks for responding, and I had understood the context to your article, but you have not addressed my key concern. My question to you was whether you felt that the IBC figures were close enough to reality to make them quotable, particularly given the fact that these are the figures fixed in the public mind.

From the sentence in your article which contrasts the estimates of 'other groups' with IBC - 'but the IBC... prides itself on working carefully with data drawn from cross-checked media reports, hospital, morgue, NGOs and official figures' - my feeling is that you take it seriously not just as 'some sort of approximation to the minimum number of deaths', but as an estimate of the number of deaths. If you did not, the numbers simply would not be worth quoting. Unless, that is, you had already discounted the other estimates (much higher) and felt that IBC was the only measure available.

Let me give an example. If I go to an anti-war demonstration, I can say with assurance that there were a minimum of 253 people attending' - because I saw that number of people with my own eyes. Or I could say that there were about 1 million people, or between 1 and 2 million people, because those are based on estimates provided by authoritative bodies.

The first is very accurate. It is certainly a minimum. But is it at all useful? And is it honest, if it is presented as the best estimate we have - which, after all, is how the public perceives it?

Thank you

JS Reply 2 (you ninny):

The IBC count is a database. The other calculations are surveys and extrapolations. Each has its value.

Jonathan

My reply 2:

I don't dispute that each has its value. But the value of an incomplete database in telling us how many deaths have been the consequence of our actions is surely limited.

Your language suggests that you believe a[n incomplete] database is more scientific than a survey or extrapolation. Experts in the field would disagree (why else is a survey rather than a database the standard procedure for estimating deaths due to war or natural disaster?). The previous government's experts certainly disagreed - as the report I linked to in my first message indicated.

I wonder at what point the database would become of little enough value to be not worth citing. 2 more peer-reviewed, scientifically accepted surveys, each of which showed the database to be out by a factor of 10 or so? 5 more? Doesn't the value of the database diminish as the results of other studies show up the extent to which it is incomplete?