Yamal and IPCC AR4 Review Comments

I was one of the more industrious reviewers for IPCC AR4. In my Review Comments, I made frequent reference to Yamal versus the Polar Urals Update, expressing concern about the rationale for using Yamal rather than Polar Urals, an issue that is once again in play. Keith Briffa was the section author and can be deemed to be the author of the responses.

As context, Review Comments for the 2nd Draft were submitted around April 2006. In late 2005, I became aware that there was additional information for the medieval period for the Polar Urals site used in MBH and Jones et al 1998 (see here (Aug 2005) for my first mention of the Polar Urals update). In Feb 2006, D’Arrigo (Wilson) et al (2006) and Osborn and Briffa (2006) were published. Both studies used Yamal and their use of Yamal was very much on my radar. See here for my first mention of the “Yamal Substitution” (plus other Feb 2006 archived posts) an While Osborn and Briffa 2006 attracted more attention, D’Arrigo et al was the more professional report. But it handled the Polar Urals-Yamal dichotomy in a very strange way – something that I’ll return to in another post.

I made a diligent effort at the time to get Science to require Briffa to disgorge his Yamal measurement data, but they refused. They argued that Osborn and Briffa 2006 did not use the Yamal measurement data, but only the chronology and I should contact the “original authors” for the measurement data. The source of the chronology was, of course, Briffa 2000. I wrote Tim Osborn and asked him for the data and he said that he didn’t have it. So I wrote Keith Briffa and he stonewalled me. I wrote back to Science rather crossly about the nonsense. Concurrently, Science was working on getting Esper’s data for Polar Urals and, after about 25 emails, it drifted in around April 2006 when the AR4 Review Comments were due.

I was also well aware that merely using the Polar Urals update rather than the Yamal series affected the Briffa 2000 composite – a point illustrated here in early June 2006, a post including the following illustration of the impact. Because the medieval networks of other studies overlap almost entirely with the Briffa 2000 network, similar results hold for (say) D’Arrigo et al 2006.

One of the issues that we’ve discussed here on many occasions is the lack of independence between the various multiproxy studies, both in terms of authorship and in terms of proxy selection. The latter lack of independence means that, if there should be some disqualifying issue with proxies used in multiple studies e.g. strip bark bristlecones and/or Yamal, numerous studies will be jointly affected. This point was endorsed by Wegman. In the 1st Draft, there was a coy mention of the non-independence – that the spaghetti graphs were “not entirely independent”:

As with the original TAR series, these new records are not entirely independent reconstructions inasmuch as there are some predictors (most often tree-ring data) that are common between them, but, in general, they represent some expansion in the length and geographical coverage of the previously available data. (1st Draft)

This seemed far too coy to me. In the 1st Draft Comment, I stated (response in italics):

6-1351 A 29:20 29:22 The non-independence should be discussed. This includes non-independence of authors and more detailed discussion of non-independence on proxy series. [Stephen McIntyre]
Rejected – the data series are discussed and point on authors not valid.

I can’t find any place where the “data series are discussed” and the point on authors is “valid” as there is substantial overlap in authors groups (Jones, Bradley, Mann, Briffa, …) The sentence was unchanged in the 2nd Draft:

As with the original TAR series, these new records are not entirely independent reconstructions inasmuch as there are some predictors (most often tree-ring data) that are common between them,…

This time I expanded on my remark on the 1st Draft, observing that it was misleading to use the term “not entirely independent”, when the overlap was actually massive. I drew particular attention to the difference between the Polar Urals and Yamal seres.

6-1167 B 30:1 30:1 You allude to the fact that these reconstructions are “not entirely independent inasmuch as there are some predictors that are common”. This is a very misleading description. For the medieval period, there is massive overlap in all the cited studies. The six series of Briffa (2000) together with bristlecones/foxtails are used in only slightly varying combinations in all of the cited studies. If there are problems with only a few canonical series (as arguably has already been demonstrated with the birstlecones/foxtails) then the entire corpus of studies may fall. Problems can be observed elsewhere e.g. the Yamal series and the Polar Urals Update have very different properties with the Yamal series being a big contributor to HS-ness while the Polar URals series has a strong MWP. The Polar Urals Update correlates better to gridcell temperature than the Yamal series and one cannot help but suspect that the decision to use the Yamal series in all studies except Esper has been done with one eye on the MWP-modern relationship.
[Stephen McIntyre (Reviewer’s comment ID #: 309-62)]

The Author Response stated that they “accepted” the comment, while denying that there was anything untoward about the repetitive use of Yamal rather than Polar Urals:

Accepted – text revised to stress overlap in early centuries of the last millennium. However, please note that the reviewer’s “suspicion” is unfounded.

In fact, they didn’t “accept” the comment at all. The coy language remained unchanged; they merely qualified it as particularly applying to the “early centuries” – a change that was totally unresponsive to the concern that overlap was massive and that it was not fairly described by the AR4 language:

As with the original TAR series, these new records are not entirely independent reconstructions inasmuch as there are some predictors (most often tree ring data and particularly in the early centuries) that are common between them,

I repeated my concern about the impact of Polar Urals versus Yamal in my comment about the AR4 characterization of Briffa (2000). The text read:

Briffa (2000) produced an extended history of interannual tree-ring growth incorporating records from sites across northern Fennoscandia and northern Siberia, using a statistical technique to construct the tree-ring chronologies that is capable of preserving multi-centennial timescale variability. Although ostensibly representative of northern Eurasian summer conditions, these data were later scaled using simple linear regression against a mean Northern Hemisphere land series to provide estimates of summer temperature over the past 2000 years (Briffa et al., 2004).

Once again, I observed that the Briffa 2000 was not robust to the use of Polar Urals as opposed to Yamal and asked that this be disclosed. This was rejected.

6-1172 B 30:7 30:7 Briffa (2000) used seven sites which recur repeatedly in the other studies. Briffa substituted a ring width series from Yamal for the updated Polar Urals series (later used in Esper et al 2002). If the Polar Urals Update from Esper is used in Briffa instead of Yamal, then the MWP in the reconstruction is higher than shown. The Polar Urals Update has a better correlation to gridcell temperature than the Yamal series (I have so far been unable to confirm the correlation to gridcell temperature of this series reported in Osborn and Briffa 2006 and suspect that it is wrong.) You need to disclose that this result is sensitive to the choice between using Yamal or the Polar Urals update. [Stephen McIntyre (Reviewer’s comment ID #: 309-67)]
Rejected – these are speculative remarks by the reviewer who incorrectly assumes that there has been a biased ‘selection’ of data and processing by Briffa versus Esper.

I raised a similar concern in connection with Esper et al (2002):

6-1173 B 30:14 30:14 The medieval network of Esper et al 2002 is closely related to that of Briffa 2000. Esper took tree-ring data from 14 sites, but only 7 extended to the medieval period. These 7 sites included 5 of 7 sites from Briffa (2000), plus 2 foxtail sites in California. Foxtails interbreed with bristlecones and may be subject to the same problems as the controversial bristlecone sites of Mann et al 1999. There is no legitimate basis for using TWO nearby foxtail sites, and probably not even one. Their relative MWP-modern level in their reconstruction does not appear to be robust to the presence/absence of these two foxtail sites.
[Stephen McIntyre (Reviewer’s comment ID #: 309-68)]Rejected – these remarks are speculative and the current text reviews published papers and is not intended to ‘second guess’ their content.

And raised the Yamal substitution again in connection with Mann and Jones (2003), again rejected.

6-1179 B 30:27 30:27 Mann and Jones use a 3-series average of Tornetrask, Yamal and Taimyr, which is not robust to the Yamal substitution. [Stephen McIntyre (Reviewer’s comment ID #: 309-74)]Noted – no revision to text necessary.

I note that the next comment following mine was accepted:

6-759 A 30:28 30:28 Missing full stop at end of line
[James Crampton (Reviewer’s comment ID #: 50-27)]Accepted

I raised the same point in connection with D’Arrigo et al 2006:

6-1184 B 30:41 30:41 D’Arrigo et al in their medieval portion use almost exactly the same network as Briffa 2000 and the other studies: Tornetrask, Yamal, Taimyr, Jasper, Mongolia (plus in their case Coastal Alaska – only one “new” series”) [Stephen McIntyre (Reviewer’s comment ID #: 309-78)]Noted – no revision to text necessary.

6-1196 B 30:48 30:48 D’Arrigo et al (2006) used only 6 sites in the medieval period, of which all but one overlap the sites of Briffa (2000) used in the other studies. They use the Yamal substitution and their conclusions of relative modern-medieval warmth may not be robust to that. [Stephen McIntyre (Reviewer’s comment ID #: 309-91)]Noted – no revision to text necessary.

Later on, AR4 contains a summary of uncertainties:

Changes in proxy records, either physical (such as the isotopic composition of various elements in ice) or biological (such as the width of a tree ring or the chemical composition of a growth band in coral), do not respond precisely or solely to changes in any specific climate parameter (such as mean temperature or total rainfall), or to the changes in that parameter as measured over a specific “season” (such as June-August or January-December). For this reason, the proxies must be ‘calibrated’ empirically, by comparing their measured variability over a number of years with available instrumental records to identify some optimal climate association, and to quantify the statistical uncertainty associated with scaling proxies to represent this specific climate parameter. All reconstructions, therefore, involve a degree of compromise with regard to the specific choice of ‘target’ or dependent variable. Differences between the temperature reconstructions shown in Figure 6.10b are to some extent related to this, as well as to the choice of different predictor series (including differences in the way these have been processed). The use of different statistical scaling approaches (including whether the data are smoothed prior to scaling, and differences in the period over which this scaling is carried out) also influences the apparent spread between the various reconstructions

This summary failed to mention what I felt were among the most relevant uncertainties in the proxy reconstruction project – the fact that similar proxies in the same region could give very different results – a fact that I thought should be clearly disclosed. They refused:

6-1205 B 31:41 31:41 You need to state clearly that proxy series from nearby sites may give very different results e.g. Yamal and the Polar Urals update. [Stephen McIntyre (Reviewer’s comment ID #: 309-100)]Rejected – this would imply a greater instability than current evidence supports.

And I also stated that they should clearly disclose the lack of robustness. Again they refused.

6-1202 B 31:28 31:28 You should add that these reconstructions are all based on a few selected proxies and that results would be different if other plausible selections were made, such as the updated Polar Urals series being used instead of Yamal or if bristlecones are not used. [Stephen McIntyre (Reviewer’s comment ID #: 309-97)]Rejected – these are the currently available reconstructions – the reviewer’s remark is a moot point.

And the overall problems with proxies – a comment that was not rejected in print, but nothing was done about it either.

6-1113 B 0:0 0:0 You should have a clear description of the potential problems with millennial proxy reconstructions: tree rings are well dated but may not be accurate thermometers; reconstructions from nearby sites may differ dramatically and overall results may be undul[y affected.][Stephen McIntyre (Reviewer’s comment ID #: 309-10)]Noted, issue dealt with with respect to specific comments on this section and the methods chapter

Finally, there was a considerable discussion about Box 6.4 Figure 1, which was a spaghetti graph of proxies (taken from Osborn and Briffa data.) The original 2nd Draft version is shown below. This graphic is exceptionally hard to read even by spaghetti graph standards. I’ve done a plot underneath showing that the spaghetti strands with the biggest blades .

Figure 1. IPCC AR4 2nd Draft Box 6.4 Figure 1 Original Caption: (a) The heterogeneous nature of climate during the MWP is illustrated by the wide spread of values exhibited by the individual records that have been used to reconstruct NH-mean temperature. Individual, or small regional averages of, proxy records used in various studies (see Osborn and Briffa, 2006), (collated from those used by Mann and Jones (2003), Esper et al. (2002) and Luckman and Wilson (2005) but excluding shorter series or those with an ambiguous relationship to local temperature). These records have not been calibrated (though all show positive correlations with local temperature observations), but have been smoothed with a 20-year filter and scaled to have zero mean and unit standard deviation over the period 800–1995.

The biggest blades in this graphic are Yamal, Mann’s PC1, foxtails, a non-full-length Van Engeln instrumental/documentary series and Yang’s China composite (which is driven by Lonnie Thompson) are Yamal, Mann’s PC1, foxtails, a non-full-length Van Engeln instrumental/documentary series and Yang’s China composite.

Figure 2. Plot of 5 key graphics in IPCC spaghetti graph.

Once again, I discussed problems with the Yamal series, criticizing its use here as follows:

6-1145 B 29:14 29:14 The beige series which has the strongest closing uptick in Box 6.4 Figure 1 is the Yamal series. When I plotted this series smoothing with a 30-year gaussian filter, I was unable to exactly replicate the uptick shown in this version. I checked the relationship of this series to gridcell temperature and was completely unable to replicate the claimed (0.49) correlation to temperature, obtaining only a correlation of 0.12. The authors here have used data from Yamal, while they used gridcell data from Polar Urals. There is an updated version of the Polar Urals series, used in Esper et al 2002, which has elevated MWP values and which has better correlations to gridcell temperature than the Yamal series. since very different results are obtained from the Yamal and Polar Urals Updated, again the relationship of the Yamal series to local temperature is “ambiguous”. [Stephen McIntyre (Reviewer’s comment ID #: 309-41)]

See response to comment 6-1143 and note that the Polar Urals and Yamal series do exhibit a significant relationship with local summer temperature.

On to 6-1143, where I had objected to the continued use of Mann’s PC1 (criticized in the Peer Reviewed Literature.)

6-1143 B 29:14 29:14 One of the most prominent series on the right hand side of Box 6.4 Figure 1 is Mann’s PC1, which uses his biased PC methodology. It is so weighted that the series is virtually indistinguishable from the Sheep Mountain bristlecone series discussed in Lamarche, Fritts, Graybill and Rose (1984). These authors compared growth to gridcell temperature and concluded that the bristlecone growth pulse could not be accounted for by temperature, hypothesizing CO2 fertilization. Graybill and Idso (1993) also stated this. One of the MBH coauthors Hughes in Biondi et al 1999 said that bristlecones were not a reliable temperature proxy in the 20th century. IPCC Second Assessment Report expressed cautions about the effect of CO2 fertilization on tree ring proxies, which were not over-ruled in IPCc Third Assessment Report. At a minimum, the relationship is “ambiguous”. In addition, I tested the correlation of this series with HadCRU2 gridcell temperature and obtained a correlation of 0.0. Osborn and Briffa say that they themselves did not verify the temperature relationship for this data. Why not? At any rate, in this example, the authors have not excluded an important series with a well-known “ambiguous” relation to temperature. [Stephen McIntyre (Reviewer’s comment ID #: 309-39)]

These comments were rejected for very weak reasons to say the least:

Rejected – the purpose of this Figure is to illustrate in a simple fashion, the variability of numerous records that have been used in published reconstructions of large-scale temperature changes. The text is not intended to give a very detailed account of the specific limitations in data or interpretation for each. Furthermore, though there is an ambiguity in the time-dependent strength of the response of Bristlecone Pine trees to temperature variability, there is other evidence that these trees do display a temperature response . Right or wrong, Mann and colleagues do apply an adjustment to the western trees PC1 in their (1999) analysis to account for possible CO2 fertilization. Other authors ( Graumlich et al ., 1991) assert that the recent rise in some high elevation conifers in the western U.S. could be explained as a temperature response (she can not confirm the LaMarche et al findings). The issue is clearly complex , as will be noted in a new paragraph on tree-ring problems that will be added to the text.

So the lead author on this section of the IPCC report was the same person, Keith Briffa, whose temperature reconstruction based on the ridiculously small Yamal sample was employed in most of the studies used for the report.

That’s outrageous! How can the IPCC expect to keep any credibility if it allows this kind of self assessment and stonewalling?

I can understand Briffa being blinded to other data once he found data that confirmed his own inherent suspicions. We all want to do something relevant that changes things.
What I can’t understand is his actions subsequent to that. The withholding of the data, the obfuscation, the censoring of comments not conducive to his message. These are not the actions of a scientist.
In another discipline, a person uses dubious data, he games the system and self-audits in order to hide that for a number of years. Eventually, it catches up with him

The discussion around Yamal has brought up again for me a question that I have had for some time. I blithely accepted the AGW story that was current until I stumbled on an article while searching for information about Daniel Defoe. http://www.cdc.gov/ncidod/EID/vol6no1/reiter.htm is an article by Paul Reiter from the Centers for Disease Control and Prevention, San Juan, Puerto Rico. Reading this article I first heard about the Medieval Warm Period and the Little Ice Age. It was an eye opener for me.

This wasn’t some flash in the pan – it was centuries. Likewise, the following Little Ice Age went for centuries. That’s centuries – not years, not decades, but centuries.

What was the basis for these assertions? Bore hole analysis? Tree rings? lake sediments? Unfortunately, no. The basis was mere human artefacts. It was the written and archaeological record. It is what the people who were there told us, directly or indirectly.

Now suppose I’m a climate scientist, and I hypothesise that an examination of tree rings in Siberia will give me a temperature proxy over this whole period. I set out my hypothesis, I collect some tree rings, I do the analysis, and I look at the results. How do I check my hypothesis. On the face of it, I would look for the effects of such well-attested events as the Roman Warm Period, the Medieval Warm Period, and the Little Ice Age. Oops, the MWP is actually particularly cold.

So, as a scientist, what do I do? Well, I throw my hypothesis out, don’t I? In fact, no. You see, I’m a climate scientist. If my hypothesis doesn’t agree with the human-attested facts, the human-attested facts must be wrong. Let’s try this.

Imagine that the sort of temperatures that have caused so much fuss in the last decade or two became the norm from, say, 1800 to the present, with no sign of receding. Now imagine that this situation only pertains around the North Atlantic. For a few centuries. Is this making sense to you? No, not to me either. I would add that it makes no sense, a priori.

The tactic I’ve adopted is a brief introductory paragraph, a link to Bishop Hill’s background blog entry followed with direct links to the “Yamal Implosion” post for further reference.

I’ve sent this to anyone I think is in a position to cover it (the last being to James Randi at the JREF, a prominent critic of pseudoscience and no mug when it comes to eliminating deliberate or inadvertant bias). Only the JREF have acknowledged so far.

(The others have been “New Scientist”, “More or Less” at the BBC and “The Naked Scientists” at the BBC)

There is no easy answer as, although fascinating and obviously important, it is not the stuff of which simple soundbites are constructed. Perhaps if McIntyre and Briffa were contestants on “Dancing with the Stars”

6-1205 B 31:41 31:41 You need to state clearly that proxy series from nearby sites may give very different results e.g. Yamal and the Polar Urals update. [Stephen McIntyre (Reviewer’s comment ID #: 309-100)]
Rejected – this would imply a greater instability than current evidence supports.
And I also stated that they should clearly disclose the lack of robustness. Again they refused.

How can Polar Urals be rejected without a justification? The fact that Esper’s Polar Urals is apparently in disagreement is the very evidence they must use when determining instability/uncertainty. And since the Briffa Yamal data seems to have one of the strongest signals this direct omission of Polar Urals becomes exceptionally poignant.

You should add that these reconstructions are all based on a few selected proxies and that results would be different if other plausible selections were made, such as the updated Polar Urals series being used instead of Yamal or if bristlecones are not used. [Stephen McIntyre (Reviewer’s comment ID #: 309-97)]
Rejected – these are the currently available reconstructions – the reviewer’s remark is a moot point.

Again, they didn’t address your very direct comment; a justification for rejecting Polar Urals was never made. Why?

If the IPCC were primarily interested in the science, there’s simply no way that the lead author of any section would feature his own work and have such power to squelch dissenting commentary by reviewers. Scientists are human beings with human egos, and the end-result of that process is fairly predictable. Of course, since the IPCC is not stupid, they must know that. What’s worse is that all of this is happening in the detailed scientific sections of the report — what happens in the process that “summarizes” the science for policy-makers is worse (although more expected, perhaps).

If the MSM are to take any interest in this then the headline they need for a sales grabbing calamity story should surely be “Climate Change plan objectives to cost our children than previously thought”, or something along those lines.

The angle would be to explain that the ‘targets’ are unachievable and indeed unnecessary having been created on the back of some limited data that, with 10 years more scientific knowledge, can be seen to be inadequate in a number of ways.

That should allow the public at large to get a feel for events and yet allow enough room for ‘experts’ to save face publically and get back to science rather than continue defending their position.

However when I take my rose tinted specs off I doubt it will work – the politicians will make sure of it.

Maybe we need to convince the Murdochs that they will make more money in a regular carbon world than a low carbon version.

Y’know none of this caveating etc. would be necessary if there were an honest attempt at quantifying uncertainty/computing confidence intervals. The only question that matters is this: why didn’t they do that? Who said “no!” The politicians, or the scientists? Or both?

Given that none of the team reconstructions provide such intervals with an explanation of how they were calculated I would go with the scientists. The politicians don’t realize that the numbers aren’t as precise unless someone hits them over the head with the old clue-by-four.

Re: bender (#15),
Good question. Thirty years ago when much of the research involved deep-sea sediment cores (fossils and chemistry) with millennial-scale intervals, there wasn’t much data available to calculate meaningful confidence intervals. Cores were scarce and widely scattered and the science was more exploratory than refining. Tree-ring chronologies came along a little later, but the habit of weak statistical rigor seems to have taken hold even though more abundant data made it possible to calculate CIs. I’d put my money on confirmation bias overriding any latent skepticism or perhaps climate scientists being influenced by geological and not biological training, rather than deliberate rejection of the idea of testing the data before you try to explain it. Politicians, OTOH, only vaguely understand the margin of error in polling data at best.

You know, it’s a good week to hit the old tip jar. Had to choose between gettin’ my Mom’s dentures fixed and keeping Steve stocked up with Ranitidine and Labatt’s. No contest! I hit the tip jar and so should you!

One: I think that if you agree that this is an important issue we should show our appreciation of Steve’s hard work by donating some money. What’s $20 in the scheme of things?

Two: Email panorama@bbc.co.uk and suggest that they cover this as past of a broader investigation in tot the corrupt nature of climate science. Plenty here for them to get their teeth into if they have the bottle to pick up the story.

Good point. I just made my $$ contribution for this excellent work by Steve.

Check out the posting by Andy Revkin over at Dot Earth. He actually included a lengthy response by Steve to Crowley’s challenge that Steve generate his own version of a tree ring based temperature record for the past few millenia. Steve’s letter is an excellent summary of the issues surrounding tree ring proxies. Considering that the link between Dot Earth and REALCLIMATENASAGISSNOAAHADLEY is intimate, I was amazed by Andy’s post.

As a plant physiologist this is the most telling statement (of Steve’s many telling statements).

“I checked the relationship of this series to gridcell temperature and was completely unable to replicate the claimed (0.49) correlation to temperature, obtaining only a correlation of 0.12. The authors here have used data from Yamal, while they used gridcell data from Polar Urals. There is an updated version of the Polar Urals series, used in Esper et al 2002, which has elevated MWP values and which has better correlations to gridcell temperature than the Yamal series.”

Correct me if am I wrong here- they took tree ring width data from one location and correlated it with temperature from another (separate) area at least 100km away – and probably alot more.

I’m sorry this screams (at best) “poor academic practice”. I’m not, as I stated a “dendro” but do have an expertise in plant responses to environmental factors. The overwhelming evidence (and I am loathe to use the word “consensus”) is that plants, trees included, respond to their local microclimate and do not to my (quite extensive) knowledge “teleconnect” to remote locations.

If the “Team” can point me to a single reference to the contrary (apart from their own), I will happily reconsider.

Re: Don Keiller (#22),
Much thanks to Don Keiller for highlighting this point for me. (So many peas, so many thimbles.) This is telling. You see how the reviewer did not refute Steve, did not deny he was correct in alleging a substitution gone wrong? (Much like Briffa’s “response” a few days back.) The reviewer implied that the difference in correlation (0.49 versus 0.12) didn’t matter; that it was significant, and that was good enough. Pea: thimble. This is dodgy perhaps to the point of dishonest. Because the issue is not whether or not the correlation is significantly different from zero (i..e the tree is a valid temperature proxy). As I have said repeatedly before, and independently of this particular revelation and investigation, the issue is possible artifical inflation of the calibration statistics during the 20th century instrumental period resulting from biased chronology substitutions. Recall my earlier concern about a possibledoubling of the coefficient? This represents a quadrupling. Needless to say, this is quite diconcerting for the effects it would have on the height of the shaft vs blade.
.
And note this is precisely the question that I asked Jim Bouldin yesterday, which he laughed off. heh heh.
.
This, to me, is the single most disturbing fact I have seen reported on Climate Audit. It is circumstantial evidence of possible malpractice AND cover-up. I am not alleging this happened. I have no proof. I am merely interpreting what is presented to me.
.
Is it possible Steve has not presented the entire review; that he has selected out specific pieces in order to manufature a story? Yes. Is that likely? I don’t know. So I will ask him.
.
Steve M: are you distorting the record in any way that could color the case against this
reviewer? Am I misled in my speculations above? Please be as honest with me as if this were a court of law. Please don’t brush off the question the way Jim Bouldin shamefully did yesterday.

Steve, you are still missing the point that the “measurement data” was not any of the Briffa reconstructions, but belonged to the Russians. The words “measurement data” should have been revealing to you. It appears that it pretty much worked out the way Eli thought it did.

If it makes you feel any better, I have no idea if the Russians would have been forthcoming.

[Jean S: if it makes you feel any better, I can assure you that there are still efforts by various parties to obtain the original data from the Russians. Keep checking in often, you’ll hear it first from here if they succeed!]

Re: Eli Rabett (#24)
Franky it does not make difference who has the data – if Briffa used it in his paper it must be provided. If he cannot provide it because he does not have the rights to the data then the papers should have never been published in the first place. And even if it was published it should not be referenced by the IPCC because papers with results thar cannot be verified are as interesting as the sunday horoscope.

Re: steven mosher (#48)
A good compromise but most asute reviewers would ask why Case A is needed if the conclusions are not affected or ask whether it is appropriate to publish a paper if the results depend on data that cannot be accessed?

Re: Raven (#25),
So what makes you think a collection of numbers you found on the internet mean anything? Did the researchers just create measurements to fit their own prejudices? Why aren’t you out there taking your own cores?

Re: Eli Rabett (#24),
That’s how you would like to phrase the issue. This is understandable, as it is something of a defense for Briffa’s nondisclosure. What is his defense against the accusation of biasing his reconstructions by making biased, and sometimes undisclosed, chronology substitutions? That this is “valid practice”, as suggested by Jim Bouldin? Shall we ask Art Wegman to have a look? No? Then name your favorite arbitrator.

Eli, regardless of the ownership of the data my point would be this. The scientific method requires that we be able to eliminate factors such as observer bias, for example. Without the release of the original data and the original methods one cannot rule out such things as selection bias. I do not need to PROVE selection bias, you need to demonstrate that there is no selection bias or any other kind of bias. The most expedient way of doing this is by revealing all the data and all the methods. From my standpoint I have no trouble saying this. I believe in AGW and I WITHHOLD JUDGEMENT on the status of climate reconstructions. You cannot say whether it was warmer or colder. Well, you can say it, but you can’t present an argument that will move me unless all the data and all the methods are available. So, basically if I read an article in a peer reviewed publication or blog or whatever I ask this threshold question: Are the data and the code publically available? If not, then no weight of any kind can be ascribed to the arguments it presents. It’s just words on a page.

Re: Eli Rabett (#27)
Steve is entitled to his opinion but something has to put an end to the endless stonewalling on the part of the “team” when it comes to providing data and it is clear that relying leaving it up to the scientists to “do the right thing” is not an option.

There is a parallel in the justice system that is worth looking at. Police are required to follow certain procedures when it comes to gathering evidence (e.g. warrents). If police fail to follow these procedures the evidence cannot be used – even if that means a guilty person goes free.

Now everyone agrees that letting a guilty person going free is a bad thing but we, as a society, have realized that without some incentive we cannot depend on the police to follow the procedures and the prospect of a guilty person going free has proven to be a reasonably good incentive for police to follow procedure.

We need an equivalent in science. If scientists refuse to provide the data required to replicate their study then the study must be ignored by any official body seeking to set policy based on science. This will mean that important information may get ignored but that is a price we need to pay in order to ensure that scientists follow the procedures that ensure the integrity of the science used to set policy.

Re: Raven (#33),
It’s called “Best Practices”. And it is up to the young dendros to outline what they think these are. It is up to people like Jim Bouldin, who comment as an authority, to do something concrete about stating what is and isn’t allowable.

Re: bender (#36)
“best practices” are meaningless without consequences for failing to follow them. Jim Bouldin can say anything he wants but it means nothing unless he can say how a dendro that fails to follow these “best practices” is “punished” for the lapse. More importantly, being “right” is no defend the consequences must apply even then.

Re: Raven (#38),
You start with a statement of principle. Then you check to see if the principles were violated. Then you worry about consequences. To become known as a violator of principle would, for some, be punishment enough.

To become known as a violator of principle would, for some, be punishment enough

I disagree. Recent history shows us that scientists can get away with almost anything and be cheered by their peers as long as they “defend the AGW faith”. OTOH, scientists can follow the principles religiously and be ostracized by their peers if their conclusions cast doubt on the AGW faith.

Now watch RC try to change the subject by posting a completely unrelated thread. Will this tactic work? Or will people keep piling on Ya Mal? We’ll see what Ray Ladbury and Hank Roberts and the other gatekeepers prefer. Watch for the phrase “move on”.

Now that they’re friends and all, maybe we can get Obama to get the Russians to hand over the data. :-)

OK. That was in jest. Here is an interesting thought. If you recall, between 2000 – 2004, the Russian govt was resisting pressure to sign on to the Kyoto protocol. The official position was that the science was not solid enough to justify Russia, which was in a recession at the time, possibly limiting its future economic growth as a result of joining this treaty. Once the ministry of energy (or somebody in high command) realized that the Russians would stand to make serious $$$$$$ through cap and trade, they signed on. But while they were resisting, they were showing climate data that was showing less warming than the IPCC claimed. I wonder if the Yamal data was kept under wraps because it contradicted the Official Russian position at the time? Also note that Yamal is a big oil producing region. Not sure if that factors in, but it’s a detail to note.

It is also common enough to publish a data description paper in a secondary journal, simultaneous with or shortly after the main scientific paper(s). That way, even somebody who uses the data for an entirely different purpose can use and reference the data giving full credit to those who collected it. Of course somebody doing similar work to the primary paper(s) would (should) reference all the papers in their appropriate places.

I know we are not supposed to speculate about motives, but reading this series of exchanges I can’t help but wonder why they wanted Steve and Ross as reviewers. It only created a paper trail that is very damaging to their claim to be conducting an objective review of the science to date.

Steve, do you think you will be asked to participate in the next IPCC assessment report?

I received the following email on Aug 1, 2005. I hadn’t submitted my name to anyone nor nominated myself. I don’t know who nominated me as a reviewer.

Dear Colleague,

You have been nominated to serve as an Expert Reviewer for the Working Group I contribution to the IPCC Fourth Assessment Report, Climate Change 2007: The Physical Science Basis. The first draft of this report will be available for expert review from Friday, 9 September 2005, with all review comments due by Friday, 4 November 2005.

Re: Steve McIntyre (#52),
The alarmists are now running around saying that Briffa politely told you years ago that the data belonged to the Russians and you had to ask the if you wanted it.
Can you provide your side of the story?

I posted up my correspondence with Science and Briffa at the time in 2006. See here.

In late 2005, I’d had a situation with Moberg where he had used a Russian and a Norwegian series (Lauritzen). He referred me to Lauritzen who refused to provide the data saying “These are unpublished data, and they come with co-authorship.” I asked Moberg to ask the Russians for consent on the data and they refused as well.

This runaround was a breach of Nature’s policies and I filed a complaint with Nature. They made Moberg sort it out as he should have done before publishing in a journal with Nature’s data policies. In the end, Moberg filed a corrigendum adding Lauritzen as a coauthor, LAuritzen’s price for putting Moberg into compliance.

It was my expectation that Science would take a similar position with Briffa i.e. if he needed consents from the Russians in order to comply with Science’s policies, it was up to him to get the consents before he published, not after. I disagreed with their decision, but left it where it was in May 2006 (remember that the NAS and Wegman reports came out soon after and I was pretty busy.) Did I leave a stone unturned? I don’t think so in any practical sense. It was my judgment that Briffa would be making the call – if he asked the Russians to give consent, I’m sure that they would have; if he didnt want them to give consent, they weren’t going to. Briffa said that he would consult with them and I didn’t hear anything further from him. I responded by writing to Science objecting to the runaround and asking them to reconsider and require Briffa to provide the data, but they did nothing.

For what it’s worth, a precisely similar situation is presently under consideration by Science in respect to Kaufman. Although Kaufman’s article stated that they used publicly available data, this is untrue. Some of the data was not public. I asked him to provide it and he said to go ask the original authors.

My position is that it’s his obligation to have had the consents prior to the article and not my obligation to chase the data after the fact. I filed a complaint with Science and it’s presently under consideration. We’ll see where it goes.

Briffa said that he would pass the request on to the Russians, seeking consent to release the data. If people are suggesting that I wasn’t diligent enough in pursuing the matter, by this time, I’d exchanged over 40 emails with Science on this and Esper. Having said that, there is an amusing backstory about the Russian data that I’ll tell in the next day or two, if people are worried that I wasn’t diligent enough.

If people are suggesting that I wasn’t diligent enough in pursuing the matter, by this time, I’d exchanged over 40 emails with Science on this and Esper.

Some people (not me, I hasten to add) might think they are justified in claiming that you weren’t diligent enough given that Keith Briffa replied to you very politely on May 28, 2006 saying

Steve these data were produced by Swedish and Russian colleagues – will pass on your message to them]
cheers, Keith

At this point, it would appear that the logical approach would have been to contact the Swedish and Russian colleagues. Did you do this?

Some people (not me, I hasten to add) might claim that you have dishonestly given the impression (note I say “impression”, not that you have actually said so directly, or even possibly indirectly) that Briffa has deliberately withheld data from you for ten years, when in fact this was not the case. And that the data was not Briffa’s to give. And that there was no publication policy in place at the time to require the data to be available. And that you were given a pointer as to who could provide the data, but you did not follow it up. At least that is the impression that some people (not me) might form, and even speculate (without justification) that whatever is going on at CA, it isn’t auditing, or science.

If I may make a humble suggestion: why not harness the power of the people who post at CA to help you coauthor a paper on Yamal for submission to the peer-reviewed literature?

I am sure that you would receive numerous offers from potential co-authors to reduce the burden on you. Suitable wiki software would enable collaborative on-line editing. If you don’t have time to set this up, I am sure that there are dozens of CA readers with the necessary expertise to help. The co-authors could assist with proof-reading, tracking down references, data, etc in return for seeing their names in print.

With this many eyes watching, it would be one of the most carefully audited papers in history, and would surely glide through the referee process. It would set a gold standard for archiving of data and software, and would greatly enhance CA’s reputation as a place where science is done. Let’s do it!

It should be possible to submit a paper on a timescale of a few months. Just think of the impact it would have! It would cite through the roof as the paper that finally showed that it is possible to choose a set of trees from north-western Siberia that show no evidence of global warming in the second half of the 20th century.

Also remember the authors of D’Arrigo et al 2006 had attempted to get this data and had been unsuccessful. If they couldn’t get the data, the only way that I was going to get the data was if Science required Briffa to provide it under their archiving policy. Once Science decided not to act, that was the end of that chapter.

Briffa 2000 was the first use of the Yamal chronology and the measurement data was archived in 2009, nearly 10 years later. In 2000, I had no involvement in climate whatever and obviously made no request at that time nor have I said so. My specific effort on Yamal began in Feb 2006. Though in summer 2005, I had asked Briffa for access to the password-protected SO&P tree ring data base and was refused access.

In this case, there is, however, a further interesting backstory regarding the Russian data which I’ll discuss in a day or two.

Some people (not me, I hasten to add) might think they are justified in claiming that you weren’t diligent enough

In response to your point that I wasn’t “diligent enough” in pursuing the matter with the Russians, in fact, I already had a version of the data from the Russians, one that I’d had since 2004. What I didn’t know until a couple of weeks ago was that this was the actual version that Briffa had used.

This is not a small point. In climate science, there can be different versions of an unarchived data set in circulation. For example, there have been a number of different versions of Thompson’s Dunde ice core data in circulation, not all of which can be reconciled.

In our 2003 consideration of MBH, even though we downloaded data from a url at his website to which we had been specifically directed and had taken the extra precaution of sending the dataset to Mann and asking him to confirm that this was the version used in MBH, Mann issued statements that we had used the “wrong” data set and a new data set materialized at his website, with the old data set being destroyed. As a result, I take extra care in requesting data from authors as used, in case differences have been introduced between the version as used and the original data.

This was a precaution that I took in connection with Osborn and Briffa 2006. At the time that I requested a copy of the Yamal data as used in Osborn and Briffa 2006, I did not know that the version provided by Hantemirov was necessarily the same as the one used by Briffa. After the Mann experience, there was every reason to take extra care to ensure that the author sent me the version as used, so that there would be no dispute at some later stage about a mistake being made about the version. Without certainty on the version that Briffa used – eventually provided in Sept 2008 – I wasn’t going to comment on the low replication. Nor was I going to spend time on potentially time-consuming analysis until I was sure that I had the data that was actually used.

In this case, it seemed pretty clear that the issue was entirely with Briffa and not with the Russians. The Russians themselves seemed not to troubled about providing the data and thus Briffa’s refusal to provide the data was not based on an inability to get consent from the Russians to provide the data, but simply his own unwillingness to provide the data.

In general, it is my view that when authors use data in a journal with data policies, if there is a permission required to ensure compliance with journal policies, they should get the permission before publication or, if they neglected to do so, get it when a request is made and then send the data. It should not be the responsibility of the requester to deal with authors not on the masthead of the publication.

In this case, the situation had already arisen with the authors of D’Arrigo et al 2006 with more serious consequences. They had requested Yamal measurement data from Briffa prior to me and Briffa had refused to provide it to them. If Briffa felt that he lacked authority to distribute the data, he should have obtained the consent of the Russians at that time – which seems to have been readily available. Instead, D’Arrigo et al proceeded without the measurement data, presumably believing that the Yamal data set was much more replicated than it proved to be, and did not carry out their usual analyses which would have revealed the low replication of the Yamal data set in 2005 or 2006 long before I eventually noticed it.

It is surprising to me how many people believe that they are supporting a stonewalling author by demanding that everyone who dissagrees with the author’s data, methods, or conclusions must themselves go out, collect data, and write a paper. If there are questions about an author’s paper, it is not up to a critic to tear the author’s work to pieces, it is the responsibility of the author to defend the data, methods used, and the conclusions.

Re: Brooks Hurd (#56), I suspect that many without a science/engineering background don’t understand that disclosure of data and methods and subsquent reproduction of result is the core of the scientific method. “Peer review” is useful but not sufficient (as we have seen) and not even necessary.

I think as non “climate” scientists/engineers become aware of what the quality of the IPCC science really is the IPCC’s influence will diminish rapidly.

You don’t need to be a rocket scientist to know/understand that if someone makes a claim, THEY need to prove it. Any one who has even written a paper in high school knows that you need to document any source.

Re: Matthew W (#76), You are of course correct. How ever I expect many non technical observers have bought into the “peer review” is the ultimate quality label, the Good House Keeping seal of approval and nothing else is necessary or is sufficient.

Not presenting data is so common a story here now. Kaufman only has 23 proxy’s to deal with. How hard can that be? Gavin put some rubbish on RC about getting data from grad students and providing results, I’m really pissed right now so I’ll snip me again.

The sight of the authors telling the reviewers that the reviewers’ comments were irrelevant still astonishes me. The IPCC Review process allowed wide latitude for Mann and Briffa to ignore inconvenient truths and there was no appeal.

The IPCC Review process allowed wide latitude for Mann and Briffa to ignore inconvenient truths and there was no appeal.

The review process may have been flawed, but the real question for me lies elsewhere: Regardless of the various review processes, why did Mann, Briffa, and others, ignore and then attack McIntyre rather than embrace him and exploit his considerable talents? In addition to being unscientific and unethical, their reaction was short-sighted. All of them could have benefited enormously from McIntyre’s technical skill.

I don’t have an answer. Perhaps they felt some gratification in their childish behavior, but, if so, it came at a very steep price: Instead of advancing the science, all they will have achieved is to reveal their incompetence, arrogance and cliquishness.

Re: TAC (#68),
Agreed. To suggest that Steve is “unqualified” is laughable. The guy has basically been working on his PhD dissertation *in his spare time* over the last 10 years. Title: “On the disappearance of the MWP from the paleoclimatology literature”. Give him the degree, for chrissakes. He’d write it up except, oh yeah, the data underlying most of the spaghetti graphs is still unarchived. (Maybe somebody could innovate a little thermometer on the sidebar which indicates what percentage of the IPCC spaghetti graph data are archived?)

There is no need for you to answer every silly inquiry about the findings you have presented. They are bright as daylight and any competent scientist can understand the issues clearly.

At some point, you have to realize that many of the people “challenging” you with obvious nonsense are just distractions, agents of the discredited attempting to occupy your time in an effort to keep you from taking the next step (i.e. Tom P).

Don’t fall for it. Don’t waste your time with them, move on.

Everyone with an IQ over room temperature can see the truth now. It is time for you to make the kill, write it up and get it published.

“… attempting to occupy your time in an effort to keep you from taking the next step”

This is a secondary, even tertiary, purpose. The prime aim is to confuse the sudden influx of unsophisticated readers, those who are attracted by the controversy but lack the background knowledge to perceive the special pleading that is served up …just read the Lorax sequences

I hope Steve will not snip this,
as I think it would serve to advance science if some of the very able readers here contacted him and perhaps dispelled his false belief that Americans are ignorant on the subject.

Considering the flap a while ago about Eric Steig’s Antarctic temperature reconstructions it was interesting to read his comments on Chapter 6. He repeatedly pointed out where the text of the chapter made statements that inappropriately minimized uncertainties and stated conclusions that were not widely accepted.

I’ll be looking more closely at his papers and his opinions since it appears that he is someone whose first priority is science and facts.

It is difficult to force an IPCC author to confront information that isn’t in the abstract of a peer-reviewed publication. When an unsympathetic author is writing the generalities that summarize the published literature, it is a waste of time to argue over the proper terms for describing the degree of the overlap in proxies/authors or the validity of one or two proxies.

What would you like to see AR5 say? How about: “Previous IPCC reports have discussed # multi-proxy climate reconstructions demonstrating that present-day mean northern hemispheric temperature is significantly warmer than during the MWP. For ALL of these reconstructions [or all # of these studies with adequately archived data and methods], replacing one or two tree-ring proxies with [equally or] more valid proxies invalidates their conclusions that current temperatures are very likely (p>0.95) unprecedented in the past millennium.” If this is what you want AR5 to say, you need to publish a paper with this conclusion IN THE ABSTRACT. There are a number of advantages to this approach. First, your paper will likely be the only publication surveying the robustness of climate reconstructions, making you the expert in the robustness field and difficult to ignore. Second, you, not the Team, will define the criteria for selecting “[equally or] more valid” proxies and for “adequately archiving data and methods” (opening the possibility of discussing the refusal to provide information). Third, you don’t have to prove that the conclusion of a Team study was wrong (which is what is required to publish a comment or induce retraction of part of a paper); you simply need to show that an alternative conclusion is possible. Furthermore, any documented demonstration of non-robustness logically negates previous claims of robustness. Finally, it is fair to request that your paper be reviewed by third parties who don’t have a stake in the robustness of Team publications.

Such a paper might also include information about how many reconstructions are no longer skillful when confidence is assessed using r2 rather than RE. (However, don’t be surprised if the IPCC chooses to ignore “inconclusive” debates about statistical methodology.)

A demonstration of non-robustness necessarily involves “cherry-picking” one or two proxy records for replacement, offering an opportunity to discuss the possibility that “unconscious cherry-picking” may have introduced these proxy records into earlier studies. You might have an opportunity to review what the literature recommends for avoiding with this problem and whether investigators, reviewers and editors have been following these practices.

Large sections of a paper on the robustness of existing climate reconstructions already exist in your posts. Writing such a paper and getting it published certainly would be a challenging undertaking, but it could prove more valuable than titillating your readers with piecemeal information or arguing with IPCC authors.

It is difficult to force an IPCC author to confront information that isn’t in the abstract of a peer-reviewed publication. When an unsympathetic author is writing the generalities that summarize the published literature, it is a waste of time to argue over the proper terms for describing the degree of the overlap in proxies/authors or the validity of one or two proxies

It seems to be an extremely odd process that an issue of the importance of AGW has a world effort from the UN based on reading the abstracts of papers published by academic researchers and a very small number of academic researchers at that.

To me the point of what Steve M does is allow thinking people some further insights into what the IPCC truly represents and whether that titillates anyone or makes anyone have that notion is really immaterial to the bigger picture.

If the problem is seen as the IPCC not being representative in its review of the peer reviewed literature what pratical value does another ignored peer reviewed paper have? On the other hand if you think the IPCC is even handed and will respond to a non consensus paper as it does the consensus then I would suggest that we have a major difference of perception that needs to be resolved first.

Nope. I’m very thorough with ITRDB as Bruce Bauer of ITRDB will attest. These are “chronology” (crn) files and not measurement (rwl) files. In any event, these are different data sets than the one used in Briffa 2000. It is precisely because there are sometimes multiple versions of data that I try to get the version of data used by the author from the author. In addition, if the author uses third party data and believes that he he does not have permission to provide the data, it is my practice to ask the author to obtain the required consent and then provide me the data so that the risk of inadvertently getting different data sets is minimized.

Re: Kenneth Fritsch (#90)
Steve’s comments were easy to circumvent with minor wordsmithing because his arguments weren’t based on the peer-reviewed literature itself, but on JUDGMENTS about how to characterize that literature. Suppose Steve succeeds in publishing a paper showing that the conclusions of “all” previous reconstructions are not robust to substitution of one or two proxy records. Such a paper will be difficult to challenge, since any demonstration of non-robustness logically contradicts all claims of robustness. AR5 is supposed to cover important developments since AR4. It will be difficult, but not impossible, for an IPCC author to dismiss in writing a peer-reviewed publication showing that previous reconstructions highlighted in AR3 and AR4 (especially the original hockey stick) are non-robust. In doing so, however, the IPCC will have created an unambiguous written record of not doing its job – reporting on the peer-reviewed literature and incorporating FACTUAL information from reviewers. The author writing for the IPCC will probably be a Team member vulnerable to the charge that he ignored Steve’s paper for personal reasons. The scandal created by ignoring such a paper will be worse than mentioning it. More likely, the IPCC will try to get away with something vague like “the robustness of some previous climate reconstructions has been questioned”. That sort of mischaracterization can only be countered by quoting exact language from an abstract.

Re: Frank (#91), It is not difficult at all, the IPCC reports habitually ignore literature they don’t like, such as Pielke jr’s work on hurricane damage being mainly due to development in coastal areas, or Svensmark’s work, or Lindzen or …This idea that if Steve simply published his results the Team would have to salute is silly. No such thing would occur.

Re: Frank (#91),
Steve has provided enough information now that there are hundreds qualified to write such a paper. Since they’re the experts and Steve is “unqualified” why aren’t people demanding THEM to write the paper? The reality is that not all of the data and methods are available to recalculate the curves in those spaghetti graphs. D’Arrigo et al. (2006) tried to get Briffa’s data and couldn’t. Why does Steve have to do what the insiders themselves cannot? Can’t you see that he is having impact through this blog, leveraging data so that others can do what he doesn’t have time for?

Re: bender (#97),
Yes. I really do like the T3 tax idea, even if the details have to change. I love the concept of finding a way to make it in people’s best selfish interest to get the science right, and removing the incentive to bias it. Of course, the challenge is finding a metric that realistically is tied to human impact ;)

If one spends months coring trees and getting bit by bugs, it is tempting to hold onto the data and use it over and over, as we see the dendros doing. This conflicts with the need to be able to verify results. My experience is that many field people will not release their data, and not just in climate science. They simply don’t bother to respond to such requests. They would rather not be cited in subsequent papers than let go of what they collected (even when it was techs who did the collecting).

That is why Steve suggested that the entire paleo matter be dropped from R4 These reconstructions help folks say now is warmer than then, but there isn’t much confidence in the result when it is put to the usual tests. The real action is in the computer climate models, a subject Steve takes up from time to time, but is not in nearly as strong a position when it comes to analysis. The models are both difficult to run and produce incredibly voluminous output when compared to the paleo reconstructions. There are certainly problems with the model output, but pinpointing the cause is not the straightforward task of spotting the most influential tree in the world once the data is published.

5 Trackbacks

[…] drawing conclusions through comparing and contrasting. For instance Steve McIntyre has posted his reviewer’s objections to the analysis of past temperatures in the 2007 assessment report, and the rejections. […]

[…] Polar Urals versus Yamal motivated many of my Review Comments on AR4 (as reviewed in a recent post here), but these Review Comments were all shunted aside by Briffa, who was acting as IPCC section […]