North and NAS on Bristlecones

BTW, I thought that the Chronicle colloquy was pretty interesting. I don’t usually get to ask people questions directly, so I appreciated that the Chronicle allowed questions through without Gavin Schmidt/realclimate censoring.

Now that I think of it, the only other question that I’ve been able to ask a climate scientist directly was to Caspar Ammann at AGU last year – what was the verification r2 of Mann’s reconstruction? Ammann filibustered about why that was an irrelevant question, but didn’t answer it.

I asked North about bristlecones (and also about error bars), but here I’ll just talk about his answer to bristlecones.

For the earliest part of the 1999 analysis, Mr. Mann’s group relied heavily on bristlecone pines from western North America. The original study noted that there were some difficulties in using such trees because of peculiarities in their recent growth, but Mr. Mann and his group attempted to quantify those problems and to work around them. The National Research Council suggested that researchers avoid using trees that are the most difficult to interpret. More-recent studies have avoided those trees and reached similar conclusions.

I emailed Monastersky about that comment saying:

The most recent studies – Osborn and Briffa 2006 and Hegerl et al 2006 – both use bristlecones/foxtails and both even use Mann’s PC1. In fact, they use separate series for bristlecones and foxtails – making 2 series of this type (out of only 12-14 total series).

He replied promptly:

With reference to the NRC report, its recommendations about the use of bristlecone pines, and recent reconstructions, I would refer you to Franco Biondi, a dendrochronologist, who participated in producing the report.

Well, there are no dendrochronological issues as to whether bristlecones were used in Osborn and Briffa 2006 – it’s a matter of fact. You could turn this question over to Price Waterhouse and get an answer.

Anyway I asked North about this in the Chronicle colloquy in a different form. Here I asked not about the properties of the bristlecones, but simply whether the panel had performed any due diligence on whether bristlecones were used in other studies (having determined that they should not be used.) Again this is not a dendrochronological question, but an auditing question.

Question from Stephen McIntyre:
The NRC Panel stated that strip-bark tree forms, such as found in bristlecones and foxtails, should be avoided in temperature reconstructions and that these proxies were used by Mann et al. Did the Panel carry out any due diligence to determine whether these proxies were used in any of the other studies illustrated in the NRC spaghetti graph?

North’s answer was as follows:

There was much discussion of this matter during our deliberations. We did not dissect each and every study in the report to see which trees were used. The tree ring people are well aware of the problem you bring up. I feel certain that the most recent studies by Cook, d’arrigo and others do take this into account. The strip-bark forms in the bristlecones do seem to be influenced by the recent rise in CO2 and are therefore not suitable for use in the reconstructions over the last 150 years. One reason we place much more reliance on our conclusions about the last 400 years is that we have several other proxies besides tree rings in this period.

It’s a typical academic answer – somewhat related to the question, but quickly going up some other trail. I didn’t ask him about CO2 fertilization; I didn’t ask him about the impact of bristlecones in the determination of uncertainties. I asked him whether the panel, having determined that bristlecones should not be used, then carried out any due diligence on any other studies to determine whether they used bristlecones – something that any engineer would have done if he determined that certain materials should not be used in a design.

Anyway, the salient part of the answer appears to be: We did not dissect each and every study in the report to see which trees were used. The next question would obviously be: Did you dissect any studies to see whether they used bristlecones, and, if so, which studies did you dissect? Of course, we know the answer to it. They didn’t dissect any studies. Getting into the data is apparently an activity fit only for "amateurs"; by contrast, the "professionals" "just sort of winged it".

North "felt certain" that the latest studies by "Cook, D’Arrigo and others" do take this "into account" – whatever that means. If "others" includes the studies actually illustrated in their study – and which North relied on in his presentation as an overlay comparison to MBH – then those other studies – Mann and Jones 2003; Esper et al 2002; Moberg et al 2005 and Hegerl et al 2006 do use bristlecones/foxtails and, in 3 of 4 cases, more than once in very small networks. Whlie a reader might think that it was outside the scope of the NAS panel to "dissect each and every study", a reasonable reader would expect them to dissect the studies that they illustrated and relied on to see if they used bristlecones. Obviously they didn’t.

What makes it worse is that, surprise surprise, they had been alerted to this very issue in our written presentation. We stated:

On an overall basis for large populations (387 sites) of temperature-sensitive tree ring series (both width and density), ring widths and densities have been declining in the latter half of the 20th century [Briffa et al, 1998; Briffa 2000]. The canonical multiproxy studies all use very small (<20, usually <10, series). Unlike the large population series, the small-sample averages tend to “support”‘? MBH98-99 through a hockey stick shape, at least suggesting the possibility of biased selection. In particular, we note that bristlecones (or inter-related foxtails), known to be a problematic proxy, but having a distinctive hockey stick shape, are repetitively selected into these small samples either directly or though Mann’s even more accentuated PC1, thus affecting, not only MBH99, but Crowley and Lowery [2000] (two series), Esper et al. [2002] (two series), Mann and Jones [2003], Jones and Mann [2004] and Osborn and Briffa [2006] (two series).

The underlying lack of robustness of a typical such study is illustrated below where the series of Crowley and Lowery [2000] are re-calculated without two problematic bristlecone series and the equally problematic Dunde àŽⳏ18 series. If these proxies are problematic and reflect biased selection, then conclusions as to the relative medieval-modern levels are not at all robust. Similar results apply in varying degrees for other multiproxy studies.

Or again in our Conclusions:

Even the principal components methodology, which has been vociferously criticized, has continued in use in prominent studies: Mann and Jones [2003], Jones and Mann [2004]. One of the networks in Rutherford et al. [2005] (coauthored by the MBH authors) was identical to the MBH98 network, including the identical PC series. Only a few weeks ago, Osborn and Briffa [2006] used the North American PC1 as one of only 14 proxies in a Science article.

Given the particular attention of our articles to the problems with bristlecones as a temperature proxy, “moving on”‘? from MBH would clearly require renunciation of bristlecones “€œ something that has obviously not happened.

So it’s not as though they weren’t told about the use of bristlecones in other studies. So they had notice of the problem, but ignored the specific notice. I guess that’s what happens when you "read a lot of papers" and "wing" it.

By contrast, the Wegman report specifically drew attention to the repetitive use of bristlecones in other studies and, in his testimony, Wegman stated that this issue needed to be specifically examined.

I think when you are at the AGU, you need to step back from the tit for tat games of this blog and make a thoughtful contribution to science. What you can show is the sensitivity of reconstructions to bcps and can give a touchstone on the possible contamination of bcps (you are not ready to stand behind dismissing them, just that they are a potential problem). The issues of NAS panel inconsistency and of quality of their work is an interesting sidelight, but not the main issue.* Examination of nature is the key objective of science.

*Note that poor work practices by the NAS panel affect both their general comments (on alternate reconstructions) as well as their statement that bcps should not be used. Given that they did not show work product and detailed rationale for indicting bcps and that you know that they did not do detailed investigation as a matter of course (did not go and stick O-rings into glasses of water), you should stop citing them as authorities on the bcp issue. It’s argumentative and childish. NAS panel report was the result of group-think and baby-splitting and the anti-bcp comments (and even the validation of your statistical criticisms) need to be taken in that context. They agreed with you without checking you.

I think that you can use them on bristlecones. Let me illustrate. In an argument, Wahl and Ammann can be relied on where they concede a MBH problem – e.g. on what the verification r2 is. But they can’t be relied on in their arm-waving conclusions. Similarly with NAS on bristlecones. Biondi was on the panel and had specific knowledge of bristlecones. They turned their mind to this issue and their finding can be cited. That they were slopy in other areas doesn’t mean that this finding is not usable.

I can’t spend every presentation arguing bcps from first principles. The NAS panel clears some underbrush.

It is a selective appeal to authority and an authority that you know has not done real hardcore thinking about the issue. If your frame of reference is in winning debates by whatever means possible (even questionable ones), fine. If your frame of reference is improving logical chains of inference, then it is wrong.

Furthermore, you yourself have been VERY careful to raise the concerns about the bcps without putting your impramateur on the statement that they are bad proxies, that you stand behind Idso, etc. Given that you have studied the issue more thoroughly then the NAS, but are more cautious in your statements, how can you honestly tout those NAS statements?

This behavior of not answering a direct question appears to be an endemic character flaw in academia. A story: my daughter started college yesterday and I told her that in class discussions if she didn’t know the answer to a question, then to admit it. One of her professors told her class not 3 hours later that if any of them didn’t know the answer, then they should “make something up.” Now, this guy is a good teacher and researcher who has had a long and productive career, but he really failed to be true to the scientific paradign with this advice. If he meant for them to speculate with supporting evidence, I can accept the answer. Just winging it, though, is to teach atrociously bad habits. BTW, this is in a “Science and the Citizen” honors course. I don’t know if this case proves the generality, but it sure does illustrate the point.

I think TCO means that making an appeal to authority where the authority (NAS Panel) hasn’t acutally examined the subject is dishonest. I say it may or may not be. If you’re not going to become an authority yourself in a subject (and you can’t do that in everything), then you have to pick the best authority available. Further, how much study a given authority has to make concerning a particular claim varies a lot. I have a nephew who is a radiologist (M.D.) who is one of the best in the business. I was talking to him and he said that generally he goes with his first glance at a chart (picture; whatever they call them). It’s a matter of being properly trained to know the signs of a carcinoma, for instance, and once you know what they look like you can be virtually certain what you’re looking at at a glance. I’ve found the same thing to be true in bird watching. One clear glance at a bird flying by and you can be essentially certain what you saw almost all the time.

So getting back to the subject, if there’s an expert on tree rings on the panel and he’s read the articles in question, he can draw his prior knowledge to make a definitive statement as to whether or not, for instance, BCPs are suitable as temperature proxies in the 20th century (and hence usuable as part of a calibration process.) The rest of the Panel, knowing that person’s expertise can then sensibly defer to his opinion. I.e., the purpose of having a Panel vs one person is to have expertise in as many needed areas as possible.

Note that this doesn’t prevent such a panel from making mistakes, but it does mean that to gainsay one of their points then requires more than just saying, “what do they know?”

So by deferring to the NAS Panel on the point of BCPs, Steve is making those who would claim they’re fine present an argument to show why they are. OTOH, Steve is also under obligation, to avoid being a hypocrite, to do the same where he disagrees with the Panel, and I for one believe he has done so. In fact his main point so far is that he already HAD done so but his points weren’t considered; e.g. that even the more recent multiproxy reconstructions still relied on BCPs et. al. to produce Hockey Sticks.

I think when you are at the AGU, you need to step back from the tit for tat games of this blog and make a thoughtful contribution to science. What you can show is the sensitivity of reconstructions to bcps and can give a touchstone on the possible contamination of bcps (you are not ready to stand behind dismissing them, just that they are a potential problem). The issues of NAS panel inconsistency and of quality of their work is an interesting sidelight, but not the main issue.* Examination of nature is the key objective of science.

TCO, I think you are confusing an auditing function for a science function. Steve M implies his function in stating the lead into this thread that:

Again this is not a dendrochronological question, but an auditing question.

An auditer can find problems and even suggest changes at a level comprehension of the field that nearly matches that of the audited participants, but in the end it is the participants (and primarily the most recognized and publicized ones) who have to change their ways in order to improve the situation. I think that is what most of us here are anticipating as we anxiously await the reactions from an engaged public and the participants themselves.

A Steve M, otherwise silent, but heroically fighting the good fight to publish some scientific article restricted to a very small part of the science that is out of his chosen field, would certainly not have drawn the attention to a more general problem as he obviously has. How many more informed citizens aware of the general attitude in the field of climatology and associated fields do we have with a Steve M in the audit mode versus a beginner in the scientific field. As an auditing function, getting the MM papers published in technical journals was and is a major accomplishment and does much to demonstrate the need for such a function.

As Wegman noted you do not do much science on blogs, but they certainly fit the mold for auditing and exposing problems of those doing the science.
I prefer a Steve M who can inject a little emotion and humor into his blogging and yet make a point as it is a welcome change of pace from the more sterile technical papers that one must read and comprehend in following the blog discussions. One can make “a thoughful contribution to science” without doing the science.

1. I’m not asking Steve to do original work. Even as a critic, he has obligations of honesty and fairness in his statements. This has nothing to do with the role of auditor. It’s just morality for anyone making statements.

2. With respect to my first para, the issue for a science conference is “are the reconstructions correct, not correct, impacted by a variation, etc.” The issue is not “what the NAS said”. TAC understood this well when he rewrote Steve’s abstract.

3. Personally I find the tit for tat to be a big distraction from consideration of who is right. A lot of blather and bravado on the Huybers response was inappropriate and not on point to examination of the mathematics. I remember being amazed when I reread the Huybers comment and saw how well he had clarified the two variables in the transform (standard deviation division and off-centering) that Steve had changed when trying to examine one (off-centering). Steve’s posts on Huybers were not very helpful in understanding what was going on. If anything, I got the impression that he was trying to muddle the water. I couldn’t be clear if it was deliberate muddling, defensive muddling or just poor issue disaggregation. But in any case, it was not right.

Similarly, the AR1 “error” Ritson discussion on this blog is troubling since you have Ross saying things like “if we conceded this point” which imply that he will not concede points when shown something is wrong, but will argue them for debate purposes. Not saying that he would, but it worries me. Similarly, those long posts are all about the sturm and drang of counter-Mannianism, but don’t give a good breakdown of what was, wasn’t done and what the math implications are so that decisions can be made. Maybe we need Huybers to come here and write down the alternate formulas!

Similarly, the AR1 “error” Ritson discussion on this blog is troubling since you have Ross saying things like “if we conceded this point” which imply that he will not concede points when shown something is wrong, but will argue them for debate purposes. Not saying that he would, but it worries me.

If you can read that into what Ross M said, I can see why you have problems comprehending the essence of the debate. You seem to find vaguely explained issues that elude everyone else and then repeat them over and over again and in very moralizing and patronizing tones. None of us are perfect but most of us here are adults and can decide with which parts of the arguments we agree or disagree.

Maybe Steve M should give you blog article space to make your point about Huybers if for no better reason than to perhaps get you to move on to something a bit more constructive than constantly attempting (and tiresomely to me) to throw Huyber into Steve M’s face. I do not think that many posters would be hesitant to comment even to the point of hurt feelings given your demeanor.

TCO – I disagree with your points, but I’m too busy right now to argue about it. I re-visited some of these issues in this post http://www.climateaudit.org/?p=798 which has some points that I’m going to use in one of my European presentations in Holland.

I made several points–it’s not clear to me what part of that post addresses what point. In any case, I understand your disinterest in discussing it with me. That’s ok. I’ll just let my points stand. Have fun in Europe.

Well, when I read Ross’ statement I thought, “Well I suppose some warmer will jerk that out of context and imply he’s actually conceding something rather than just trying to find an area of common ground to argue from, but I didn’t suspect that someone ostentively on the skeptic side would make such a logical error. That’s what it is, you know. It’s like if a mathematician presented a reductio ad absurdum proof and then got knocked because his starting premise was clearly wrong.

My concern is not so much that he is in error, but that he is (maybe, possibly) not conceding it when maybe he should. I have seen other failure to concede a point in the past and where something had to be dragged out of Steve or Ross.

Steve has before made the point of the weakness of conditional arguments in a trial defense. Ross’s statement is a conditional argument of defense.

In any case, what a lot of sturm and drang that does not even help to explore the meat of the issue. I suppose I could go research it, dig into it, read Ritson’s stuff, etc. But what a waste for Steve not to make his posting on it explanatory. I come back to how good the Huybers clear description of the data transform was.

Riddle me this, DD: doesn’t it strike you as odd that Steve spends so much time on the “Mann did it too” aspect rather then the “it’s ok to do it” aspect? (To be honest, I don’t think he takes a firm stand one way or the other as to which method is right…one more reason why the Ross post struck me as troubling.)

TCO,
Steve has the very difficult task of working at two levels simultaneously:
1. A specialist attacking a particular domain problem (or set of problems),
2. A generalist interested in solving the more fundamental problem of disclosure, peer review, etc.
He can’t do it all, all the time, and do both equally well. Choices have to be made.

You want him to do (1), but clearly (2) is the more fundamental problem, where quick victories also can be scored. As the competing sharks in the specialist shark-tank out there pick up on (1), (2) is going to become increasingly attractive. The optimal rate of transition to (2) is going to depend on the rate at which others move in to adopt (1).

Please stop repeating yourself. Or better, get Steve to create you a “TCO-pounds-on-Steve” thread (like he did for “Dano’s agriculture” and “bender’s hurricanes”, which were boring people to death).

A lot of the continuation of argument comes from people arguing with my assertions. (Not all of it…some is my fault.)

I did not get your point on the 1 versus 2 thing and I’m not sure that it adresses all my concerns about gaps in Steve’s logic or candor. But running with the concept anyhow, I’m not sure that seguing from an actual examination of a charge to “he did it too” is really a case of one versus two. And wouldn’t you think as a scientist, engineer, logician, etc, that one should examine the charge itself?

I know that my points can become tiresome and am willing to leave again. I do think that as long as this is an open forum/community, that arguments on places where Steve got it wrong should be allowed. I think it is unfortunate if Steve’s work is not critically examined…if key questions like “when Steve lists 4 separate criticisms are they really only 2 separate issues” or “what is the numerical effect of a specific flaw” are not asked and answered.

Here is another example where Steve keeps “getting it wrong” in my opinion: he talks about how off-centering mines for the “bad” apple of the bcps…for how it grabs non-climate factors. But off-centering doesn’t mine for non-climate. It mines for a SHAPE. The matrix algebra has no way of knowning if bcps are CO2 or sheep or dry lake bed contaminated. Those are separate issues. When Steve confounds those kinds of things, it makes me cringe. And it’s not just dumb guys like me, who don’t know vector algebra. It’s guys like Zorita, who quite calmly tell Steve, “I don’t know if the method mines for bad apples or not, what is the definition of one”? (I’m sure he is looking for a spec of a shape.)

P.s. I bet if Wegman were here, he would back me up. And Martin Ringo who knows vector algebra out the ying yang, has been very kind and responsive to my questions, to places where I have pushed Steve for logical defense or for numerical description.

he talks about how off-centering mines for the “bad” apple of the bcps…for how it grabs non-climate factors. But off-centering doesn’t mine for non-climate. It mines for a SHAPE

TCO, the rest of us understand that from the start. In general, it mines for a shape, but in those particular circumstances it mines for a particular shape that is evidence favouring a particular hypothesis. You, on the other hand, seem to need every little detail spelled out for you. Steve is not making an error, he’s just being brief. So instead of accusing him of an error, be gracious and ask for an explanation. You’ll look a lot less silly, dear friend, and waste less bandwidth.

I have seen him confound issues before in discussion. I think said confounding is distracting to proper examination, to proper onion unpeeling, to reducing 60 to 2*2*3*5. Examples of places where said poor or muddled (deliberate or sloppy, not sure) have occurred are in PC1 versus reconstruction and on off-centering versus standard deviation dividing. It’s not JUST that he might be making an error in communication, but that we need to know for certain what the specific assertion is so that it can be validated.

Chill Willis. I said it is one OR the other. (I guess it could be some of each.) I have covered some of this stuff in the past. but you ask, I answer: In terms of recent examples, there is the tendentious appeal to an authority, that Steve doesn’t really believe in for a statement that he’s not even willing to get behind himself (on the bcps).

If I were to follow Ken Fritsch’s ground rules, which I really like, the safest option for me would be to ignore your postings from now on. That would put you in the same category as Dano and Bloom, but if that’s what you want …

Re #28: bender, any reasonable definition of ignoring me would have to include not constantly invoking my name or linking to a prior comment of mine (which you did earlier today, the very same day on which you now claim to be ignoring me). But that’s OK, this is probably just another area for which you lack Bayesian priors.

I don’t think you’re playing games, TCO. I’m just at a loss as to what to say to clarify #22. You’ve been told a dozen times by others than me that your constant harping at Steve M to publish, do this, do that, is annoying. There are only so many hours in the day. He can’t do it all. Choices have to be made. Can I be any more clear?

I’ll ignore you alright. Not because you’re playing games. But because you’re a time sink.

My comment was that your comment while interesting, didn’t seem to address what I was aiming at. But I think I get your point now, while still disagreeing. On Steve’s choices: sure, he can do whatever he wants. But if there are going to be multiple 1000-word posts and tens of comments on an issue (e.g. AR1), I think it is reasonable for me to criticize the lack of illumination and even the misdirection involved in looking at all the side issues, rather then the main issue, itself.

To me, that’s Mannian. That’s why I left for a while, and probably will again. It felt like pinning down a snake to get Steve to come to grips with the difference between PC1 mining and reconstruction impact, and he STILL tends to muddle that difference to try to overstate an impact. And plenty of the gallery here goes away with an overstated view of the impact. That’s wrong. And since he doesn’t write (as a rule) finished papers/arguments, but instead uses the blog for communication, it’s fair to judge him on what he says here.

It was incredible, how many posts he wrote about Huybers critique without clarifying the very good point that Huybers had made about Steve changing TWO variables while trying to make a statement about the impact of changing ONE. And would ya believe it, changing TWO gave a bigger impact then changing ONE. That bugs me.

I also think that Steve’s case against Mann’s statistics has been one of detail…so it is reasonable to discuss said details. In addition, Steve has faulted Mann for rhetorical tricks rather then clear examination of issues, so it is unfortunate to see Steve wax tendentious.

On Ross’s statement, yes, I can read that in. But, as stated, it is a concern, not an accusation.

I read his post and thought he was employing a standard and often used phraseology to make his point. To be able to read something nefarious into such common language (and you do this often with Steve M) is a form of innuendo, and vague at that, that serves no useful purpose in furthering the discussion. Part of my problem with your critiques is that they are not sharp and clear, to me at least, and so if I am looking for criticism of Steve’s and Ross’ work you would not be my first choice.

Group think is to be avoided and, if that is what you are about then I say you have good intentions, but that is best accomplished in my view by clear and thoughtful questions without a lot of inferences attached.

re: #23

P.s. I bet if Wegman were here, he would back me up. And Martin Ringo who knows vector algebra out the ying yang, has been very kind and responsive to my questions, to places where I have pushed Steve for logical defense or for numerical description.

It is statements like the above that make me perceive that you are very much full of yourself and this seems to guide your responses at this blog. I often hear from you that you pushed Steve M to do this or that and that TCO is seldom wrong. Wegman would back you up on what — not appealling to authority? What are these places specifically where you have pushed Steve for logical defense or for numerical description? And how did all of this add to or influence the discussion? If you could be more specific and clear in your responses perhaps I could take your efforts more seriously.

Steve has talked about the danger of conditional defenses. Ross’s comment was a conditional defense!

In addition, Steve’s post talked about “I don’t concede it for a minute”. Well, does he contest it? Which is right and why?

In addition, Steve has a bunch of (correct) comments about “if we make the change, the effect still remains but at a lower level”. So fine. That is a conditional defense also. If Ritson is right, admit it. If not, show why he’s wrong. If you’re unsure if he is right or not, admit that (and discuss the issue itself).

TCO,
You seem to be the kind of person that says what he means and means what he says. You don’t what there to be any misunderstanding when you communicate with others; you want to express your ideas in such a way that your audience understands it the same way as you do. I know, because I have a similar mindset. However I have found that no matter how precise, exact, and complete you express youself, your audience will never see it in exactly the same way as you do. Their own biases, experiences, and knowledge domain ensure that. As a result you just come off being tedious, reptitive, and ultimately boring. I’ve learned to resist this impluse myself and it has really helped my relationships. Think about it. Make your points and move on. Debating is fine, but try to avoid endless arguing.

Im not haranguing you…I’m discussing to detail. Yes, you made a post (Ritalin) about the issue, but I don’t think it went into detail on how he is wrong. It was all about “even if he were right”. Furthermore, when you say “he is wrong” are you referring specifically to the issue of AR1 assessment of proxies in recent years? Or to the earlier issue on his differencing.

Jumping half way into a debate that I haven’t read all of, hope I haven’t misunderstood anything here…

TCO,

The Ritalin comment was not so much targetting the AR1 coefficient estimation, but the fact that Ritson was demanding a response from Wegman within around four weeks or so when the hockey team were guilty of dragging out responses much longer than that.

The AR1 coefficient issue was covered in more depth here. The problem Steve identified with the Ritson method compared to the conventional method was that in the event of model misspecification (ARMA(1,1) vs. AR1) the Ritson method gave very wrong results, whereas the conventional test is more robust. The inconsistency between the Ritson method and the conventional method tie in almost exactly with this type of model misspecification.

One Trackback

[…] exemplified by North’s response to my online question to him during a Colloquy (threaded at CA here) Question from Stephen McIntyre: The NRC Panel stated that strip-bark tree forms, such as found in […]