IPCC WG1 Publication deadlines

I’ve previously discussed IPCC WG1 publication deadlines in the context of Wahl and Ammann [2006], where the authors seemed to make last-ditch efforts to comply with IPCC WG1 publication deadlines, but ironically failed to comply with the letter of the deadlines. There have been some other high-profile "late-breaking" articles (Osborn and Briffa [2006], Wahl et al [2006], Hegerl et al 2006) which seemed to have been published with an eye on the IPCC 4AR buzzer. I thought that it would be interesting to collate IPCC WG1 publication deadlines and see whether these buzzer-beating shots got off in time. As I am bound by WG1 confidentiality, I am dealing here entirely with a hypothetical situation, continuing a discussion which I started well in advance of the release of the Second Order Draft. Readers should not conclude that any of these studies are or are not mentioned in the Second Order Draft. This is all hypothetical.

In my previous post on the matter, I linked to the IPCC WG1 policy statement on publication deadlines at the WG1-UCAR website here . The version available in February has been replaced by a new version ; the previous version may be found in a web archive here. The differences are not material, but both versions are shown below. [Update: The new version mentioned here is available only at a web archive now – see here.

Three publication deadlines are mandated. The first benchmark is May 2005. In the first order draft, reviewers are entitled to cite unpublished literature provided that it is supplied in draft form so that TSU can make it available to reviewers.

First Version: May [2005]. Second Lead Author meeting, probably May 10 to 13, Beijing, China. This meeting considers comments on the zero order draft and writing the first order draft starts immediately afterwards. Note. Literature to be cited will need to be published or available in draft form by this time. Copies of unpublished literature should be sent to the TSU so they can be made available to reviewers if requested.

Present Version: When authors start to write the first draft for the AR4 they should have final or draft material for all work that is being cited in front of them. In practice this means that by May 2005, papers cited need to be either published or available to LAs in the form of a reasonably accurate draft of what is expected to be the final publication.

When the first draft of the AR4 is sent out for expert review, copies of any cited papers not already published must be made available to reviewers on request. This means that LAs need to ensure that drafts of any such papers are sent to the TSU before or at the same time as the chapter drafts, for which the absolute deadline is August 12. Such drafts will be made available to reviewers in confidence solely for the purposes of the review and are not to be copied, cited or re-distributed by them.

The second deadline is December 13-15, 2005. To be cited in the second order draft, literature needs to be "published or in press by this time". This has allowed 7 months for articles which may have only been available in draft form in May 2005 to have wended their way through the journal review process. This deadline would not permit the inclusion of articles published in March 2006, unless they were "in press" by December 13-15, 2005.

First Version: Third Lead Author meeting, December 13 to 15, in Christchurch, New Zealand. This meeting considers comments on the first order draft and writing of the second order draft starts immediately afterwards. Note. Literature to be cited will need to be published or in press by this time. Copies of literature not available through normal library sources should be sent to the TSU so they can be made available to reviewers if requested. Meeting of the TS/SPM writing team December 16 in Christchurch, New Zealand

Present Version: When the second draft of the AR4 is written authors need to be sure that any cited paper that is not yet published will actually appear in the literature, is correctly referenced, and will not be subsequently modified (except perhaps for copy editing). In practice this means that by December 2005, papers cited need to be either published or “in press”.

The third deadline is "late February 2006". This is a drop dead date in which TSU must hold "final preprint copies" or else TSU will ask authors to "remove any reference" to it.

First Version: When the second draft of the AR4 is sent to Governments and experts for the second round review, the TSU must hold final preprint copies of any unpublished papers that are cited in order that these can be made available to reviewers. This means that by late-February 2006 if LAs can not assure us that a paper is in press and provide a preprint we will ask them to remove any reference to it.

Present Version: When the second draft of the AR4 is sent to Governments and experts for the second round review, the TSU must hold final preprint copies of any unpublished papers that are cited in order that these can be made available to reviewers. This means that by late-February 2006 if LAs can not assure us that a paper is in press and provide a preprint we will ask them to remove any reference to it.

Obviously, to comply with the February deadline, it’s not enough to simply have obtained journal acceptance by the end of February. That was supposed to have been done by December 2005. One can also contemplate cases where an article could have met the February deadline, but not the December deadline. For example, if an article was not "published or in press" in December, even if there is a preprint by the end of February, it has not met the second deadline.

In an active field like climate science, what if there was some late-breaking paper that was important to include? Does it make sense to talk about a "buzzer" or to adhere to strict policies? In this case, it does for a couple of reasons.

One potential risk in any article is that it merely represents a "point of view". The editorial wars at wikipedia provide object lessons in "POV". If an article is "late breaking" and its scheduling is seemingly tailored for IPCC 4AR, then one could reasonably be wary that it is a POV article. Several of the high-profile "late breaking" paleoclimate articles (Wahl and Ammann 2006; Osborn and Briffa 2006; Wahl et al 2006) are obviously controversial. Even if you think that Wahl and Ammann 2006, for example, is a terrific article, you know that it is not the last word on the topic and that it is going to provoke a Comment, perhaps a lengthy Comment. It’s part of an ongoing controversy and, simply because the ink is barely dry (in fact, there is no ink yet), no one assumes that it’s the last word in the controversy.

If the late-breaking results were critical of publications by IPCC lead authors, then one might reasonably conclude that the IPCC lead authors were bending over backwards to ensure that IPCC fairly represented the literature and one might condone a little leniency.

In most walks of life, there’s usually a reason for administrative policies and one of the main reasons is to avoid accusations of favoritism. In a municipal contract, if a qualified contractor gets a contract without due process, there is always grounds for suspicion. But if the mayor’s nephew gets a municipal contract without due process, then no one really cares how meritorious the nephew may be, the contract is tainted.

Briffa is an IPCC lead author and Osborn and Briffa [2006] is one of the late-breaking articles. Ammann is a close associate and co-author with Mann, an important lead author of IPCC TAR, and Otto-Bliesner, a lead author of IPCC 4AR. If lead authors are circumventing publication deadlines in order to provide exemptions for their own articles or for articles by close associates, then I do not believe that any leniency or exemptions should be granted. IPCC lead authors and their associates were well aware of the deadlines. If their late-breaking articles did not strictly comply with IPCC WG1 publication deadlines, then too bad. IPCC lead authors should not be permitted to circumvent WG1 publication deadlines to defend the work of their close associates or to promote their own work

17 Comments

The obvious failing is that a paper may be submitted, peer-reviewed and published yet may be utterly wrong which can be demonstrated by one or more papers attempting to reproduce the results. The IPCC process is obviously scientifically incestuous and non-independent.

Where is the external audit? Where is the paper trail? Who watches the IPCC thimble?

John A., this is done by governments. If you have a problem with what’s going on, I’m sure there’s an appropriate offical you can contact in the British government. I doubt they’ll accept anonymous complaints, though.

I’m not aware of any governments or governmental agencies that audit the IPCC. If anyone knows of one, I’d love to see the results of any audits they’ve done. Heck, even the name of such an agency would be a step in the right direction, and if it’s in the US government maybe I will complain. No “anonymous” red-herring BS can be thrown my way.

This seems to include plenty of latitude for an audit if someone were able to convince a member government to ask for one. Both the UK and US are currently voting members of UNEP. Let me know how it turns out.

If by “Seems to include plenty of latitude” you mean, does not in any way discuss any form of oversight policy or audit whatsoever, and absolutely in no way does it even hint of a method of inquiry into the UNEP, then possibly your on to something.

Sid, have you been following the news about the UN in the last year or so? Security council members, in particular the US, have quite a bit of power when it comes to oversight of UN operations. I’m confident that the US could have an audit of the IPCC if it really wanted one. Come to think of it, the CCSP process currently being undertaken *is* a scientific audit of the IPCC, plus there’s the separate NRC panel process with a specific focus on paleoclimatology. What more do you want?

As to the CCSP and NRC panel, you’ve got it wrong again. They are reviewing studies that are also included in teh IPCC. It is not a review of the IPCC. To use a poor analogy, it’s kind of like when “Road and Track” magazine does a review of the latest Chevrolet Corvette, and you are saying that’s the same as a SEC audit of GM.

Your completly and totally off base, and are applying totally irrelevant information.

Present Version: When the second draft of the AR4 is sent to Governments and experts for the second round review, the TSU must hold final preprint copies of any unpublished papers that are cited in order that these can be made available to reviewers. This means that by late-February 2006 if LAs can not assure us that a paper is in press and provide a preprint we will ask them to remove any reference to it.

(emphasis mine)
I note with interest that (as of today), the TSU *still* does not hold a “final preprint copy” of at least one of the unpublished papers cited in the 2nd draft of the AR4. The (seemingly final) preprint copy of this paper obtained from an author’s web site is approximately 50% longer than the copy held by the TSU, includes roughly 30% more citations, 40% more main tables/figures, and so forth. If all those changes and extra material aren’t germane to the IPCC 4AR, one would think that the journal’s editor would also have been able to get rid of a lot of it before publication…

A great argument for sticking to the letter of the process is to remember the kerfuffel arising from modification of the last summary for policy makers AFTER it had been signed off on by reviewers. This loosey goosey IPCC episode resulted in a protest petition by I forget how many thousands of people. You would think they wouldn’t want a repitition of it.