Tuesday, September 04, 2007

Amer and Scheuren

One of the more annoying tendencies of Lancet defenders is that they refuse to recognize the breadth of academic criticism against the Lancet papers. Over time, I hope to gather some of that criticism together. Here is one example.

Several ASA statisticians have gone to Iraq since 2003—Jana Asher and Mary Gray, to mention two. An article by Asher about her experience appeared in the December 2006 issue of Amstat News; Gray’s work in Iraqi Kurdistan can be found in the April 2007 issue. So, the involvement of statisticians in Iraq is not unusual.

The recent controversy over the estimated Iraq war deaths that appeared in the October 11, 2006, issue of The Lancet has drawn considerable statistical attention to the situation in Iraq, as the results in the article did not appear credible. As mentioned by Gray, a follow-up to The Lancet piece occurred at a February session of the Washington Statistical Society (WSS), organized by Wendy Rotz, with talks by Asher and David Marker. The jury is still out on The Lancet results. Statisticians must assess the process used, not the outcome. But there is a lot wrong with the process, leading to the conjecture that the results are most probably wrong.

Indeed. Do these authors have standing for believing that the Lancet results are "probably wrong?" Judge for yourself.

Safaa Amer holds her PhD in statistics from Oregon State University. ... Fritz Scheuren is a vice president for statistics, who works with Amer at NORC at the University of Chicago. He put together the What Is a Survey? booklet some years ago. In his 2005 ASA presidential speech (March 2007, JASA), he advocated a pro bono role for statisticians.

I look forward to reading Daniel Davies thoughts on why Amer and Scheuren are just another two members of the ever-expanding list of anti-Lance cargoculters. Time to get those boots on!

2 Comments:

trane said...

Well, Mr. Kane, the two authors certainly have STANDING, as you indicate. But they present no REASONS for believing what they believe. So it is difficult to judge, really. They may have a valid argument to back up their claim, but it is not presented here. This is important if we are to appreciate the ’breadth of critique’ of the Lancet study, that you urge people to recognise. Being CAPABLE of making an academic argument within a particular scientific discipline is not the same as MAKING it. And besides, advocating for a new study of mortality in Iraq is much in line with the authors of the Lancet study.

Anyway, it would suit you immensely FIRST to make some distinctions, and SECOND to start giving answers rather than conjectures and loose claims.

The distinction you need would be 1) Scientific arguments about the Lancet studies vs. Hack arguments of the Lancet studies.

You may find it easier to get recognition from all those diehard Lancet-supporters who won't listen to an argument, if you set a high bar yourself as to the forms of critique of the study you will accept as valid.

This should make you cut connections to people who have not read the study carefully, and therefore criticie their own invented study, not the actual Lancet study.

A further distinction (2) is between VALID and INVALID arguments. For instance, if you find your own critique of the Lancet study lacking (following the discussions at Deltoid), then you would label it SCIENTIFIC BUT INVALID.

In this way, your critique will not be in the form of:

”And Malkin says, and Spagat says, and some statisticians say, and Fumento says.... and even though some of them are proven wrong, they can’t all be, because they can’t.”

If you choose to not follow this form, and instead stick to academic ARGUMENT (not only conjectures), you might have something valuable to contribute.

The first step to make contribution is to clarify what you mean when, on Deltoid, you write:

"There is nothing to retract! Again, now is not the time to re-open the endless thread, but the big message of the paper still stands, although many details will be fixed in the next version."

I would ask you kindly to state in the simplest language possible1) What big message is left from your paper? And what is the warrant, that is the grounds for believing, this big message?

2) What is the implication of this message for the value we should ascribe to the L1 study?

About Me

I am an Institute Fellow at IQSS at Harvard. I organized a panel on mortality in Iraq for the August 2008 Joint Statistical Meetings in Denver, Colorado. Participants on the panel includes Safaa Amer of NORC, Michael Spagat of Royal Holloway College, University of London, Mohamed Ali of WHO and Olivier Degomme of WHO Collaborating Centre for Research on Disaster Epidemiology.