U.S. Climate Change Science Program Workshop

I’ve had an abstract accepted for the U.S. Climate Change Science Program (CCSP) Workshop, "Climate Science in Support of Decisionmaking," to be held November 14-16, 2005. My abstract is entitled: "More on Hockey Sticks: the Case of Jones et al [1998]".

The workshop is described here. Here is my abstract (which moves on from MBH):

More on Hockey Sticks: the Case of Jones et al [1998]

Abstract. Multiproxy studies purporting to show 20th century uniqueness have been applied by policymakers, but they have received remarkably little independent critical analysis. Jones et al. [1998] is a prominent multi-proxy study used by IPCC [2001] and others to affirm the hockey stick shaped temperature reconstruction of Mann et al. [1998]. However, the reconstruction of Jones et al. [1998] is based on only 3-4 proxies in the controversial Medieval Warm Period, including non-arms-length studies by Briffa et al. [1992] and Briffa et al [1995]. We show that the Polar Urals data set in Briffa et al [1992] fails to meet a variety of quality control standards, both in replication and crossdating. The conclusion of Briffa et al. [1995] that 1032 was the “coldest year” of the millennium proves to be based on inadequate replication of only 3 tree ring cores, of which at least 2 are almost certainly incorrectly crossdated. We show that an ad hoc adjustment to the Tornetrask data set in Briffa et al [1992] cannot be justified. The individual and combined impact of defects in the Polar Urals data set and Tornetrask adjustments on the reconstruction of Jones et al [1998] is substantial and can be seen to have the effect of modifying what would otherwise indicate a pronounced Medieval Warm Period in the proxy reconstruction. Inhomogeneity problems in the Polar Urals and Tornetrask data sets, pertaining to altitude, minimum girth bias and pith centering bias will also be discussed.

Over at davidappell.com, one of the posters commented on my AGU abstract and postulated about that conference:

I don’t know how papers get handled at these conferences, but if it’s possible to simply bar the door to him I suspect that’s what will happen.

I hope that doesn’t happen, as I enjoyed the AGU conference last year.

but they have received remarkably little independent critical analysis

This is where it goes wrong isn’t it? This one phrase condemns near every referee for every paper you’ve tried to trash as, well, not independent – thus biased. It also implies only you are independent, there’s a thing eh?

I must say if I’d ever refereed anything climate based (and I haven’t) and I read that I’d just think ‘Well, thanks pal for that gross insult to my integrity. If that’s your attitude I can understand the responses you seem to get from the climatology profession’. And why not?

#3 “…I can understand the responses you seem to get from the climatology profession’. And why not?”

Why not? Very simply, if the “climatology profession” could refute the critical analysis of M&M they would. Since they can’t they prefer to bar him at the door.

What the climatology profession fails to understand is that their perversion of the scientific method is going to eventually blowback on the profession in a very serious way. Barton is just the beginning of the blowback. If all the studies cited by the IPCC received the same scrutiny as MBH then many more Piltdown Mann studies would be exposed.

Peter, a point that I’ve consistently made is that journal peer review is a very limited form of due diligence. The issue isn’t bias or integrity; it’s just that referees don’t go through these studies in detail. Because the review is so limited, that’s why I’m so insitent on the view that data and code should be archived for paleoclimate – on the same basis as is now standard in econometrics. It’s simply adopting a “best practices” standard.

I am also not saying that I am the “only” independent person. Why would you put such words in my mouth? However,I am unaware of any critical, independent analyses of this study – if you disagree, perhaps you can cite one for me.

2. I don’t read the comment as indicting peer reviewers as “non-independant” but as not doing detailed analysis. This is almost certainly the case, since peer review RARELY does deep analysis of the data (auditing). So I don’t see this as a “slam on the individual reviewers”, but just comment on level of detail involved in peer review in general. And comment was not only directed to peer reviewers (in fact you have to read in to even come up with that). Since peer review is anonymous and somewhat secret, a more reasonable interpretation is that there were few “audits” in PUBLISHED LITERATURE.

3. This is a conference specifically on policy decisions based on science, not on the science itself. It is (even more than normally) germane to dig into amount of double checking of work (methods of the community) as a topic for this type of meeting.

Peter, We are dealing here with differences of opinion. The advancement of science comes from differences of opinion, not from uniformity.

You stated that “every referee….” is “not independent – thus biased.” You are inferring the bias from what Steve wrote. Clearly if all the reviewers of an article are selected only from a group of people that shares the same opinions as the author, then these reviewers might overlook details which people of differing opinions might not overlook. It is human nature to push forward an argument or an article which supports an agenda with which you agree. This does not mean that all the reviewers are biased, however one could argue that they were not as critical as they might have been. One might also make the point that the skill sets of the reviewers was unsufficiently broad to encompass all areas upon which the article is based.

Multi-proxy studies are based on the statistics of data from various sources which are in some way related to temperatures. The correlation of some of this data to temperature is very close. The data from other proxies does not have the same direct correlation to temperature, in that other factors besides temperature might effect the data. Every author of a multi-proxy study is dealing with the same variables. The key to a multi-proxy article’s conslusions is the statistics applied to the enormous amount of data in the study.

If the statistics are not sound, then neither are the conclusions. It is the statistics which are key to critically evaluating multi-proxy studies. When an author of a statiscally based article refuses to release their data, they are not allowing independent verification of the statistics. Esper’s group includes no one with a degree in mathematics or statistics. This does not mean that the statistics his group used were incorrect. However, when Esper or other authors do not post their data on accessible sites, it prevents independent critical analyses of the statistics which are the very basis for their articles.

I am uncertain, Peter, but perhaps you know how many statisticians have reviewed multi-proxy articles written by Esper, Mann, Jones, Crowely, and other.

I concur heartily with your main message about importance of math/stats soundness in this type of work. Well stated. I would hesitate to say that none of the Esperites are not strong stats/math types. They may have undergrads in math/science/physics and then have specialized with the Ph.D. For instance Mann (not part of this group) has what would seem to be strong math credentials (mathemical physics type stuff) if you look at his earlier degrees.

It is a lovely abstract, but in terms of the science I would roughly characterise that abstract as the equivalent of a combination of a garlic wreath, stake through the heart and decapitation. These are arguments that cannot be ignored. There may be spirited discussion !

I think Peter Hearnden misses the point in his comment on independent audit. The point of these reconstructions are that they are whole new methodology for climate science. Large scale collations of several data sets are being brought together, and analysed en masse. This is new technology for the climate scientists; this is its novelty.

But in science, there are many, many examples where people have brought together new technologies, and misunderstood one of the new fields. The infamous “cold fusion” fiasco was a failure of the scientists to understand the difficulties of low level heat measurements. Slightly more relevantly, it is within living memory that medics used to be the ultimate arbiters of the value of medical treatment; a particular treatment was deemed by a consultant to “work in my professional judgement”. Large scale clinical trials demonstrated that individual judgement was often wrong; a statistician/ epidemiologist was required. This is now such received wisdom, that papers going to some medical journals have mandatory, separate and independent review just of the statistics.

This is all part of the normal process of how science moves on. Are M&M right ? We will see.
yours
per

Actually maybe peter has a good point and that there is a way to rewrite the abstract to be less inflammatory (I think the flames come from jumping to conclusions, but still need to think about people who may do that) and more matter of fact. (while still hitting the main point).

Per, much of the work in “evidence-based medicine” originated in Ontario. Guyatt, one of the leading authors, is actually from a squash playing family as well. For others, if you google the term evidence based medicine, you will see many interesting articles, much of it focusing on review articles. The paralellism of a medical review article to a multiproxy study seems pretty striking to me. Much concern has been placed on bias in review articles, especially towards recommending intervention. It’s not hard to think of parallels. I read an interesting profile of a NIH administrator who decided to commission actual statistical studies of medical treatments in the late 1970s, with real statisticians working with the doctors. He was drummed into exile. Nowadays the drug companies seem to have some really excellent statisticians working for them. So it’s not impossible to contemplate a future in which good statisticians are involved in climate work.

I read Steve’s abstract with great interest, of course, and put a different interpretation on the text than Peter Hearnden did in #3. I thought that “independent critical analysis” would have come from papers in the literature.
I hope the titles and texts of other relevant abstracts can be posted somewhere in due course.
Remarks like “non arms length” and “almost certainly incorrectly cross-dated” are mild enough. These must be pretty bland compared to statements at the early meetings on “continental drift”.
Steve, congratulations on having the abstract accepted.
Roger Bell

Peter,
Maybe you would be comfortable spending billions of dollars and perhaps changing the course of civilization on just the word of a few scientists, but I’m not. Claims of this magnitude require critical review by multiple reviewers in multiple disciplines before any sweeping policy changes are made using them. These reviews mean complete access to all the data and computer programs used.

Indeed. Further, a basic skeptical axiom is that extraordinary claims require extraordinary evidence. A claim that human activity is the dominant factor in current climate variation fits the category of extraordinary- at the least a situation that we have never encountered before. This alone argues for extra effort in “auditing”.

I’ve said before, re #3, a scientist needs to be a little thick skinned. We should not take anyone’s word for it that everything is OK, and need not be examined closely, least of all the word of the people that did the work or their friends, colleagues, or coauthors. The work needs to stand independently of its makers, and withstand strict criteria of replicability.

Replicability, in my eyes, doesn’t mean that everyone of a group of collaborators gets consistent answers, it means that someone absolutely not involved in the original work needs to be able to reproduce the findings. If they have the competance to do such a study, and cannot reproduce the results, the work is in doubt. Not wrong, not deceitful, but it can reasonably be doubted. This then should stimulate debate, generate new work and resolve the matter. I think the science is being stifled by unwillingness to be forthcoming with data and computer code.

I remember a line from a song, and I don’t mean to offend anyone, that said “Two men say they’re Jesus. One of them must be wrong”. I’m not sure how you resolve that issue, but I think the audit concept is a way to deal with scientific disputes. Nobody’s going to like it, but the data and computer programs need to be there to look at. If it’s hidden, it’s suspicious. I don’t see anyway ’round that.

Perhaps Mike Hollinshead could also comment on ulcers. The consensus theory for many years was that ulcers were caused by stress. Only recently have they concluded that they are caused by bacteria. Early on in the study of ulcers, some physicians said ulcers were caused by bacteria, but they were derided.

Re #22. The consensus here for many months is that ALL (every damn one as far as I can see) the recent recons are so flawed they should be discarded and insulted. Early on in this places existance a few people, like me, made post to the effect that actually the recons are, broadly, right but we have been consistently derided (infact, usually worse). So you think we’re going to be shown to be right? Thanks :)

Nice try Peter, but no cigar. The recons shouldn’t be “discarded and insulted”, rather insulted first and then discarded. The point is Peter, have you read any of the posts that deal with why so many of the reconstructions are scarely worth the paper they are written on, or perhaps it is fairer to say that the conclusions drawn from the papers are not worth etc. Haven’t you yet accepted that MBH98 is fatally flawed. At the very best its conclusions should be “that past temperatures have varied and the 20th century values may be relatively high, but within the limits of our statistical treatment we cannot tell if this is unusual or not.”

And you should cheer up, we haven’t derided you merely for asserting that the recons are broadly right, but for being unable to support that assertion with data when the various recons are challenged. For example you have made no data based attempt to defend MBH, for example on whether the Bristlecone Pines should be included as a temperature proxy, but merely asserted that it is correct. So you can avoid derision easily, simply argue on the facts, you might even find enough support in areas for people to agree with you, should you come up with convincing and supportable arguements.

For me the words of Steve aren’t the handed down truth. I don’t, as you do, unquestioningly accept that MBH (and all the other recons) are fatally flawed and I wont unless or until the evidence is more convincing than the writings of two Canadians (who’ve nothing constructive to offer as an alternative) and a bunch of their utterly convinced supporters.

For me that evidence isn’t good enough. However, there is plenty of evidence to indicate now is both very warm and historically warm and that the present warming might very well accelerate considerably, and damagingly, this century ‘thanks’ to our activities. Again, you’ll probably deride my view…

Peterf, you are so set in your own mindset that you seem incapable of imagining any other way to think. You seem extremely keen to accuse everyone as follows: “For me the words of Steve aren’t the handed down truth. I don’t, as you do, unquestioningly accept that MBH (and all the other recons) are fatally flawed and I wont unless or until the evidence is more convincing than the writings of two Canadians (who’ve nothing constructive to offer as an alternative) and a bunch of their utterly convinced supporters.” Just for the record, unlike you I didn’t come with the an unquestioning acceptance of a point of view. Instead I came to read, and not just to believe, but to follow up and to come to my own conclusions. You, by contrast, came with a set view that MBH was correct, and you refuse to engage with any data produced that undermines that view. Unlike you, I am willing to change my mind.

You see, you say “For me that evidence isn’t good enough. However, there is plenty of evidence to indicate now is both very warm and historically warm “. This is what much of the evidence produced by Steve on this site deals with, and I think that although that view could be correct, adequate evidence to assert that it is has not been produced. The reconstructions you seem so fond of are not well founded. You believe what you want to believe, fine, but if you post here be prepared to be challenged to introduce supporting data.

You could quite easily turn that on its head – It is an extraordinary claim that doubling the amount of CO2 in the atmosphere will have no effect on global temperature. It requires extraordinary evidence.

Re #27 – I think that the impact of 2xCO2 is a real issue. I have no personal views as yet as what would be the impact. It is precisely because I think that it is an important issue that I think that scientists who are studying these matters should meticulously archive their data and methods.

RE #23. Peter, while I criticize many of these reconstructions, I make a conscientious effort to see how they work. I don’t simply “insult” them, although, sometimes, merely describing what’s actually going on is pretty demeaning to them. While I occasionally lapse into use of adjectives, mostly I write pretty objectively.

Re #29. Steve, do you think any of the recons, post HH Lamb, offer insight of worth into past climate? I don’t think you do.

TBH this place is about finding fault not finding out. While science is, in part, about replicability, it’s also about finding out, about inquiry, about research. Anyone coming here hoping to find out about past climate is going to be very disappointed you, as you’ve oft stated, have no view on that. No one here is doing the hard work of finding out, just the easy bit, finding fault.

Re#26. Ed, I don’t think MBH is correct anymore than I do Moberg 05. Both (and the rest from, and including, Lamb onwards) offer insights into the past and build upon each other. None is correct, otoh, none is wrong (despite all the thousands of harsh words here), none merit being discarded and none are fatally flawed.

Peter, why do you think, at our present state of knowledge, that it is possible to produce a proxy record of the earth’s temperature accurate to a couple degrees or less? That’s the obvious assumption you’re working under or you wouldn’t complain that Steve is just criticizing rather than providing his own climate reconstruction.

I’d submit that the null hypothesis is that no such reconstruction can be produced.

I think that we can slowly learn some things (and maybe eventually find a killer proxy). I just don’t think that we should overemphasize certainty if we don’t yet have the skill. Also don’t think that we should bias things by cherrypicking.

Peter, it’s not that I started out “trying” to find fault with these studies. Initially I just wanted to see how they worked. Now I think that the cited multiproxy studies all depend on a surprisingly narrow set of indicators, which are poorly constructed. My main point is that you can’t rely on any of these things.

The Lamb view of climate history was largely thrown out because of the supposed quantitative-ness of tree ring reconstructions, as advocated in Hughes and Diaz, 1994, still cited. I’m entitled to examine these quantitative methods. It’s impossible to examine them without finding fault. I’m not “trying” to find faults; they are lying there in plain view.

My background is in math. People can and have pointed out flaws in math proofs from time to time, without themselves being able to prove the theorem. Their inability to themselves prove the theorem doesn’t invalidate their finding the flaws in a supposed proof.

Re #31. Dave, how accurate do you think we can know past climate, and at what times in the past, and across what areas, are various accuracies possible? Or do you think such reconstructions impossible? Do you even think we can say anything about past climate pre thermometers? I suspect the latter is where you stand – perhaps you’ll answer those questions?

Why do I think it’s possible? I think that because a lot of scientists think it’s possible and I don’t think I know better than them. You clearly do think you know better than them, yet, as far as I’m aware, you haven’t published any relevant science that we can get our teeth into to check your credentials. As I said, criticism is easy, actually producing some new science leaves you open to criticsm – that’s probably why you (?) Steve and the rest never produce anything new.

TCO #32, I think, as often, you are within that area marked ‘broadly right’.