Dear All, Hopefully, the last time. After considerable review of what the authors have done, and some feedback from others, I've revised and streamlined my "response". The authors convolution of incommensurate scaling factors with the posted Mann et al eigenvectors, resulting from their use of a different instrumental temperature data set and inconsistent normalization convention (standard deviation of detrended series was used in Mann et al while non-detrended gridpoint standard deviations have been used by the authors to unnormalize the Mann et al eigenvectors--this leads to a spatially-variable biased enhancement of variance) appears likely to be the culprit. Its clearly an error. There may be several others yet uncovered, but this alone certainly invalidates what they have done. The attached, hopefully final version of my "response" should suffice. Thanks, mike

Date: Mon, 27 Oct 2003 15:50:38 -0500 To: mannatXYZxyzginia.edu From: "Michael E. Mann" <mannatXYZxyzginia.edu> Subject: Re: Fw: New Study Questions Kyoto Global Warming Data Cc: asocciatXYZxyz.net, Gabi Hegerl <hegerlatXYZxyze.edu>, Jonathan Overpeck <jtoatXYZxyzrizona.edu>, Keith Briffa <k.briffaatXYZxyz.ac.uk>, Malcolm Hughes <mhughesatXYZxyzr.arizona.edu>, Mike MacCracken <mmaccracatXYZxyzcast.net>, Phil Jones <p.jonesatXYZxyz.ac.uk>, Ray Bradley <rbradleyatXYZxyz.umass.edu>, Ben Santer <santer1atXYZxyzl.gov>, Steve Schneider <shsatXYZxyznford.edu>, "Lonnie G. Thompson" <thompson.3atXYZxyz.edu>, Ellen Mosley-Thompson <thompson.4atXYZxyz.edu>,Tim_ProfetaatXYZxyzberman.senate.gov, tom crowley <tomatXYZxyzan.tamu.edu>, Kevin Trenberth <trenbertatXYZxyz.ucar.edu>, Tim Osborn <t.osbornatXYZxyz.ac.uk>, Tom Wigley <wigleyatXYZxyzker.UCAR.EDU>, apetsonkatXYZxyzironmentaldefense.org, "Jim Salinger" <j.salingeratXYZxyza.co.nz> Dear All, One small correction upon further reading--they did appear to use subsets of eigenvectors corresponding to those nominally used by Mann et al (1998) upon further reading of their description. However, the eigenvectors aren't the same, because the surface temperature dataset is a different one from that used by Mann et al! So the eigenvectors #s that they used are not the same as the corresponding eigenvectors used by Mann et al (1998). Again, because they didn't apply the objective criterion themselves (they appear to have simply used the same nominal eigenvectors, but of a different(!) temperature dataset), they didn't appear to discover the problem... mike At 02:38 PM 10/27/2003 -0500, Michael E. Mann wrote:

Dear All, The link below has been passed along to me by a colleague (whose name has been expunged to protect his identity): Based on the description of data and method provided, which the authors claim is a complete (!) description [this is important], I have confidently concluded that they have neither used the same data (instrumental or proxy) but, more problematically, the same method as Mann et al (1998). In fact, there is a huge problem in the statistical calibration procedure they used, which omits an essential step used by Mann et al (1998) to protect against statistical overfitting in the presence of an increasingly sparse proxy data network back in time. 1) They used the wrong instrumental data set. Mann et al (1998) used the older version of the CRU instrumental surface temperature data set that dates back to 1854; they apparently used the newer version of the data set that goes back to 1856 which Phil published and updated to in the mid 90s--the data sets are actually significantly different in places. They differ significantly, for example, in where they have missing data. Phil can provide people details. Thus, the eigenvectors are assured as being different, and the missing data are different. It would have been surprising if they had found otherwise! 2) They used the wrong proxy data. The authors apparently used an excel spreadsheet version of the MBH98 data that my associate Scott Rutherford had sent them. It appears that the data got shifted and scrambled a bit in the process of being converted to an excel spreadsheet or upon being downloaded or opened. This would explain the numerous transcription errors the authors find in the file. Of course, we used the uncorrupted data in our study. These ascii versions of the data have always been publicly available on our computer "holocene". Had the authors used the (correct) ascii series on the data set, they wouldn't have encountered transcription errors. From what I But they appear to have had a corrupted version of the data. The authors then describe an elaborate effort to download suitable approximate versions of the proxy data series they couldn't get ahold of. In many cases, these appear to be substantially different versions of the proxy series than the ones used by Mann et al (1998) But again, I don't think this matters. 3) They did not implement the Mann et al approach!! From the description of the method provided, it appears that the authors skipped the essential step of (1) applying an objective criterion (i.e., Presidendorffer's Rule N as used in Mann et al, '98) to determine the optimal size N of the subset of the full (16) candidate instrumental principal component series to retain in the calibration of the proxy data and (2) optimizing the calibration resolved variance with respect to all subsets of the leading PC series of size N. These crucial aspects of the procedure were clearly layed out in Mann et al (1998), and is perhaps one of the most essential steps---it is only the application of this objective criterion that prevents an obvious statistical overfitting problem--the authors *always* appear to use a subset of all 16 PC series! However, the criterion used by Mann et al (1998) dictated the retention of a maximum of 11 PC series, only a few PC series prior to AD 1600, and only one prior to AD 1450. So the authors appear to have tried to fit 16 PC series to the reconstruction from AD 1400-AD 1450, when an objective criterion would only dictate 1! This is a really basic statistical error, and its likely this massive overfitting that is responsible for the wild behavior in their reconstruction prior to about AD 1600. Can't beleive they made such a basic error? See for yourself: [1]http://www.climate2003.com/computations.html Any statistical climatologists worth their salt would have picked this up. But I believe the paper wasn't even reviewed! 4) Their result is, not surprisingly, wacky The resulting reconstruction they show, with an enormous warm anomaly during the 15th and 16th centuries, looks like nothing ever produced in any objective estimate of past Northern Hemisphere temperature trends that I'm familiar with. I guess they could term this the "Renaissance Warm Period", but of course no other model or empirical Northern Hemisphere temperature reconstruction looks anything like this. Frankly, the ridiculous result should have let them know they did something wrong.

Mike, This seems to be the HTML version of McIntyre's paper. (I got this unsolicited from Gene Avrett, Soon & Baliunas's boss at Harvard.)

---------------------- Original Message ----------------------- From: "Eugene H. Avrett" To: Date: Mon, 27 Oct 2003 12:09:22 -0500 Subject: article in Energy and Environment ---- Dear , You may be interested in the article by McIntyre and McKitrick just published in Energy and Environment which questions the validity of the Mann et al. (1998) study that provided the basis for the claim that 20th century warming is unprecedented. See [2]http://www.uoguelph.ca/~rmckitri/research/trc.html. Yours sincerely, Gene Avrett