The Dog That Didn't Bark

This is the title of a famous Sherlock Holmes story and not intended as a slight to any individual.

Take a look at the Review Comments for AR4 Second Draft Chapter 6 online here.

While I was reviewing these comments, I noticed that there are no reported comments on chapter 6 from Caspar Ammann, one of the major participants in the paleoclimate debate. In the Chapter 6 Review Comments, there are many comments about his article, but none by him. In the IPCC roster., he is shown as a Contributing Editor but not a reviewer [Note – this has been modified in light of #7 below].

I’ve just gone through the exercise of searching all the First Draft and Second Draft Review Comments for any on-the-record comments by Ammann and found none. Perhaps the online Review Comments do not include comments by reviewers if they are Contributing Authors to another chapter. If anyone feels like perusing the Review Comments for examples, I’d be interested. This is a bit surprising in a way, since Susan Solomon’s review comments were submitted through the IPCC review process and are on the record, so why wouldn’t Caspar Ammann’s comments on chapter 6 also not be on the record?

Terminology from a then unsubmitted paper turns up in the Chapter Author replies to Review Comments. Compare the terminology in the answer to Review Comment 6-735 (which would have been final around Aug 4, 2006) to the language in Ammann and Wahl, which, as we recently learned, was submitted on August 22, 2006. Some language points track exactly and do not occur elsewhere in the literature. So there is no doubt that Ammann made written comments to the Chapter Authors of Chapter 6, but these are not on the record despite IPCC policy which states:

All written expert, and government review comments will be made available to reviewers on request during the review process and will be retained in an open archive in a location determined by the IPCC Secretariat on completion of the Report for a period of at least five years.

Update: Here is an amazing parallel between the Replies to Review Comments for chapter 6 and Ammann and Wahl 2007. The Reply to Review Comment 6-735 stated:

Rejected – the text is based on the authors’ interpretation of the current literature (and all papers cited are within current IPCC publication deadline rules). The text gives a balanced view.

Please note the following –
The MM05d benchmarking method is based on an entirely different analytical framework than that used by MBH98.

MBH used the standard method in climatology of making a random time series based on the low-order AR characteristics of the target time series during the calibration period. here the N. Hemisphere mean. This random process is repeated in Monte Carlo fashion and its skill in replicating the actual target time series is evaluated according to any measure of merit in which the investigator is interested.

MM’s method instead uses the full order AR characteristics of one of the proxies used in the reconstruction to create pseudoproxies in a Monte Carlo framework. These are then input into the reconstruction algorithm along with white noise pseudoproxies for all the n-1 remaining proxies. This is, in theory a statistically meaningful procedure, which asks what kind of apparent skill is available in the reconstruction simply from one proxy’s noise. However this procedure is not general and would need to be repeated for each proxy set to be examined. Also, it would need the subjective choice of which single proxy should be modelled according to its red noise characteristics each time.

Finally, it does not take into account that some of the verifications seen as “skillful” are associated with very poor/exceedingly poor calibrations, which would be rejected on first principles in real world reconstruction applications. This consideration indicates that the 0.51 threshold cited by MM is actually, at least somewhat, overstated.

I defy anyone to show me any support for these comments in peer reviewed literature as at August 4, 2006. But here are some quotes from Ammann and Wahl 2007:

Standard practice in climatology uses the red-noise persistence of the target series (here hemispheric temperature) in the calibration period to establish a null-model threshold for reconstruction skill in the independent verification period, which is the methodology used by MBH in a Monte Carlo framework to establish a verification RE threshold of zero at the >99% significance level.

Rather than examining a null model based on hemispheric temperatures, MM05a,c report a Monte Carlo RE threshold analysis that employs random red-noise series modeled on the persistence structure present in the proxy data (note, noise here is meant in the sense of the ‘signal’ itself, rather than as an addition to the signal). … RE performance thresholds established using this proxy-based approach have the disadvantage of not being uniformly applicable; rather, they need to be established individually for each proxy network.

Furthermore, the MM05c proxy-based threshold analysis only evaluates the verification-period RE scores, ignoring the associated calibration-period performance. However, any successful real-world verification should always be based on the presumption that the associated calibration has been meaningful as well …

Interestingly, the first paragraph cited above caught my eye in Ammann and Wahl and I asked him for a supporting reference, justifying this argument. To which, Ammann, a federal employee, answered:

why would I even bother answering your questions, isn’t that just lost time?

Update Jan 2009: This particular mystery was resolved by the Climategate Letters, which contain off-the-record correspondence between Wahl and Briffa.

Ponder this. In the Second Draft Review Comments, “Ammann” occurs 64 times (!) but Ammann doesn’t comment himself. How odd is that?

Caspar Ammann is listed by IPCC as an Reviewer, but he didn’t make any review comments. I wonder how many other people are listed as reviewers, but didn’t make review comments.

Now I think that there’s overwhelming evidence that Ammann communicated with Chapter Authors – I’ll post up some evidence later today. It’s just that he seems to have done so off the record and not to have done so on-the-record as Ross and I were required to do.

I had a look at the AR4 WG1 annexes, and it seems Caspar Ammann is listed in Annex II (Contributors) on page 955. However, he is not listed in Annex III (Reviewers) which start on page 969.
Could that be why?

#7. Ammann was a Contributing Author to chapter 9 (but not chapter 6). So you have a point and I’lll modify the post accordingly. But I think that it raises an interesting question. Did Ammann submit written comments on chapter 6?

IPCC procedure states:

All written expert, and government review comments will be made available to reviewers on request during the review process and will be retained in an open archive in a location determined by the IPCC Secretariat on completion of the Report for a period of at least five years.

Even if Ammann were a Contributing Author of chapter 9, if he submitted written comments to the Chapter Authors of chapter 6, shouldn’t those be included in the open archive? Susan Solomon’s comments are on the record. It is inconceivable that Ammann didn’t submit comments on chapter 6. Shouldn’t they be on the record?

#10. We’re getting to the main issues. Still laying a little groundwork. This is a blog after all. Glad that your interest is whetted.

I’ve added a section to the above post showing the remarkable parallelism in language between the Reply to Review Comments for Review Comment 6-735 and language in then unsubmitted Ammann and Wahl. The Chapter Authors asserted that they were giving a “balanced” view of the literature, while relying on unpeer reviewed opinion from Ammann to supposedly rebut Review Comments.

I thought with that title it would foster a different kind of debate. SM, I think these people see you as adversarial, however listening to your radio interview, you laid out your viewpoints very carefully to not bring on that kind of response, and I noticed that you don’t give up meaty sound bites for the media to run away with. It could be the just the way these people read the tone out of your emails, have you ever tried phoning, instead of writing?

All written expert, and government review comments will be made available to reviewers on request during the review process and will be retained in an open archive in a location determined by the IPCC Secretariat on completion of the Report for a period of at least five years.

Maybe that’s the catch…his communications weren’t written and therefore weren’t required to be available and archived.

I believe what we are seeing is the belief among some scientists (and it is not just in climate science) that there is an inner circle of anointed scientists who can dispense with certain formalities required of lesser mortals. Formalities like proving your point, citing authoritative references, and taking criticism seriously. Once you reach this exalted station, you can speak from authority and everyone is supposed to listen to you. Steve M. violates the code by not bowing down with sufficient obsequiousness.

I believe what we are seeing is the belief among some scientists (and it is not just in climate science) that there is an inner circle of anointed scientists who can dispense with certain formalities required of lesser mortals. Formalities like proving your point, citing authoritative references, and taking criticism seriously. Once you reach this exalted station, you can speak from authority and everyone is supposed to listen to you. Steve M. violates the code by not bowing down with sufficient obsequiousness.

It is on record that the novel hero was nearly named “Sherrington Hope” but was finally cast as “Sherlock Holmes”. Numerous Google references. Also note that Toronto Public Library has an extensive collection of Doyle’s works, eh Steve?

A CG2 email from Tim Osborn expressing a somewhat disdainful attitude toward my “”proof” that Ammann corresponded directly with us and not via the official review comments process, and thus his correspondence is apparently not included in the official set of comments/responses”
The correspondence was between Ammann coauthor Wahl rather than Ammann himself, but the surmise was 100% correct. This makes me question the degree to which Osborn (and perhaps even Jones) were aware of the surreptitious contact between Wahl and Briffa and that Wahl had re-written IPCC language that had been sent to external reviewers to suit his own ends.

1701. 2008-05-26 10:12:07
______________________________________________________
cc: t.osborn@
date: Mon, 26 May 2008 10:12:07 +0100 (BST)
from: “Tim Osborn”
subject: FOI stuff
to: k.briffa, p.jones
Hi Keith and Phil,
I see on ClimateAudit that there are now many posts on Wahl & Ammann, its citing by Chap 6 of IPCC and how this fits with the publication deadlines etc., late changes to the IPCC publication deadline rules (a memo from Manning), plus “proof” that Ammann corresponded directly with us and not via the official review comments process, and thus his correspondence is apparently not included in the official set of comments/responses:

McIntrye: “I’ve added a section to the above post showing the remarkable parallelism in language between the Reply to Review Comments for Review Comment 6-735 and language in then unsubmitted Ammann and Wahl. The Chapter Authors asserted that they were giving a “balanced” view of the literature, while relying on unpeer reviewed opinion from Ammann to supposedly rebut Review Comments.”

David Holland comments that he will follow up some of these issues:
“Incidentally, the email address in the Manning memo is ipcc-wg1@al.noaa.gov and must be subject to US Freedom of Information Law. I will be asking some questions this side of the pond, hopefully someone in the US will also ask a few.”

So I’m sure that Holland will be pursuing or even expanding his FOI request! I can’t remember what emails I do or don’t have, but it would be useful if we do have the one from Wahl/Ammann confirming that their article received final acceptance, to demonstrate that we followed the rules concerning the deadlines for material that we cited.
Tim

3 Trackbacks

[…] exists outside of the climategate mails. The mails, merely confirm what Steve McIntyre had already figured out. The mails of course add some background that strengthens the case, including the hacker’s […]

[…] as his “borrowed (stolen)” language was noticed in a CA post of May 24, 2008 (The Dog That Didn’t Bark), leading me to postulate surreptitious non-open non-transparent contact between Wahl and Ammann […]