Mann et al 2007 Precipitation Teleconnections

Judith Curry suggested that we talk a little about Mann et al 2007 available here . I noted its publication last summer when Jean S and UC made some remarks about it. It has an extensive SI with code here and, in fairness to Mann, there is a serious attempt at documenting his work in the SI. At the time, I noted that Mann et al 2007 continued the use of his incorrectly calculated principal components – something that seems absurdly stubborn in view of the clear statements by both the NAS panel and Wegman that the calculation was erroneous. It also reflects poorly on the JGR referees that they should have allowed this incorrect method to remain in discussion and on the more extended community, including JEG and Judith Curry, that no one has seen fit to hold any of them accountable for it.

Today, I noticed something very interesting in his SI, that goes back to MM 2003 and demonstrates the fanatical resistance to admitting even the slightest error. In MM03 here, we observed that there was an exact match between the precipitation series in Mann’s gridcell 42.5N 72.5W (in New England) and the GHCN historical series for Paris, France. I summarized the match at the time with the phrase:

The rain in Maine falls mainly in the Seine.

Here’s an excerpt from MM03 ( a paper with which Mann is familiar)

Subsequent to MM03, we observed that the location of all but one precipitation series in MBH98 was incorrect. We reported the problem to Nature. In Mann’s Corrigendum, he merely corrected the incorrect attribution to Bradley and Jones 1993, saying that the data came from “NOAA” but offering no hints as its provenance within that organization and not admitting that the locations were incorrect. In the 2004 Corrigendum SI, he left all the incorrect geographical locations uncorrected.

In the new SI, there is a table comparing each proxy to the corresponding gridcell and I thought that it would be interesting to see what happened with all these incorrectly located precipitation series – including whether Mann et al 2007 had stubbornly perpetuated the erroneous location. The table is here .

Sure enough, there’s not been a single change in any of the incorrect locations. The rain in Maine still falls mainly in the Seine. How bizarre. Mann then proceeded to calculate correlations between each precipitation series and an incorrect gridcell location – often wildly incorrect. Thus the Paris precipitation series was compared to New England gridcell information, Toulouse precipitation to a South Carolina gridcell and a precipitation series near Philadelphia to the information from the gridcell containing Bombay! A severe test of teleconnection.

I did a quick re-analysis in which I calculated correlations of the 11 MBH98 precipitation series to the 72 GHCN (ndp041 vintage) precipitation series that started before 1830. For each MBH series, I identified the GHCN series with the highest correlation and then made a new table showing the lat and long of the identified series and the MBH series. About half the series have correlations of 1 and are firmly identified. Other series have correlations of around 0.8-.9 and the exact provenance of the series remains unknown although the locations are undoubtedly near the ones shown here. To my surprise, I actually figured out how Mann screwed up his precipitation coordinates. Take a look below and see if you can figure out the pattern.

Station Id

Correlation

GHCN Site

GHCN Lat

GHCN Long

MBH Grid Lat

MBH Grid Long

4327900

1

MADRAS/MINAMBAKKAM

13

80.18

12.5

82.5

7240811

0.913

WEST CHESTER 1W

39.97

-75.63

17.5

72.5

0763000

1

TOULOUSE/BLAGNAC

43.63

1.37

37.5

-77.5

0765000

1

MARSEILLE/MARIGNANE

43.45

5.23

42.5

2.5

7250706

0.821

NEW BEDFORD

41.63

-70.93

42.5

7.5

0715000

1

PARIS/LE BOURGET

48.97

2.45

42.5

-72.5

1123100

0.836

KLAGENFURT-FLUGHAFEN

46.65

14.33

47.5

2.5

1151800

1

PRAHA/RUZYNE

50.1

14.28

47.5

12.5

0365701

0.838

OXFORD

51.7

-1.2

52.5

12.5

0310202

1

EALLABUS

55.6

-6.2

52.5

-2.5

0316001

1

EDINB.OBS./BLACKFORD

55.9

-3.2

57.5

-7.5

The first row is located OK. But the MBH lat-longs for the 2nd row shouldn’t be there. All the coordinates for rows 3-11 should be moved up one row to rows 2-10. I presume that he intended to have a Bombay precipitation series in the data set, but forgot to actually include it. So a series is shown in that location when none actually comes from there. While a Bombay precipitation series might provide useful information about the monsoon, one feels less confident that precipitation data from near Philadelphia, interpreted as Bombay precipitation, will provide a lot of useful information on the monsoon, but I guess we should never underestimate the power of teleconnection.

This is the third time that I’ve seen a goofy collation error in MBH materials. In the original proxy data set that Rutherford directed me to, all the PC series were off one year. This was the famous data set that Mann subsequently said was the “wrong” data set and deleted, reproaching me for actually relying on the data set at his website to which Scott Rutherford had directed and which I’d even asked Mann to confirm that this was the correct data set. (A new archive suddenly materialized in Nov 2003.)

Then in Rutherford et al 2005, there’s a similar collation which we discussed at the time, in which, as I recall, his temperature series are all displaced one year incorrectly relative to the proxy collation.

When I first observed that the rain in Maine falls mainly in the Seine, I hadn’t considered the possibility of teleconnection. I just thought it was an error (and I still think this.) JEG describes Mann et al 2007 as a “sophisticated” analysis. But because Mann has not corrected the geographical locations of his precipitation series, all the calculations showing correlations to the gridcell location for these series are incorrect – despite this matter being brought to his attention long ago.

All in all, I feel that it detracts somewhat from the explanatory value of Mannian teleconnection theory if data sets could be located on incorrect continents without any affect on the results.

Given that you have identified the mistake, you might
consider writing a comment and submit to the journal.
It would only take a few hours or so to generate and
will probably go through the system very quickly
since it is a matter of fact.

However, a more extensive comment along the lines of “if these location errors have no effect on the result, one must ask whether the result is predetermined by the method rather than the data” might have a chance at pressing some buttons.

Good catch. We know all kinds of collation errors can and do happen, that is why the most astounding thing is the reluctance to double check after this type of issue has been brought to someone’s attention. Hubris?

I am utterly amazed at how you manage to keep slogging through this stuff and still preserve your sanity and sense of humor. It must be like debating the residents of the Alzheimers wing at the local nursing home on a daily basis. After a while I would think it would drive you bonkers. I’m nearly there and all I do is read the posts. I think your statistical knowledge is much less important to this endeavor than your temperment.

I skimmed through the paper and it appears primarily be a long-winded defense of MBH98 and 99 piling on with more sciencey sounding stuff. If he fixed the previous errors he couldn’t achieve his goal of keeping his previous work on a pedestal.

From what I can tell, adding in the sciencey parts no matter how incorrect they may be is what keeps it from being pseudo-science in JEGs evaluation.

Someone here with more skills than I should replot his distribution graphics, maybe with a color key showing proxies off by at least a continent.

1 Perhaps Mann had a reason for deleting the phantom Bombay Proxy?
2 I do hope Dr Curry and JEG are updating their students. Wouldn’t want them to leave with the wrong idea.
3 I have a song for Dr Mann, here is “He’s King Midas in reverse” with versions by the Hollies, CSNY, and a version by the The Posies – live @ Criminal Records, which (although I hadn’t heard it before) sounds very appropriate. Here are the lyrics.

Folks, here we have an opportunity here in front of us, let’s seize the moment.

I agree, that Nature will probably put Steve’s point in the circular file, it’s easy to ignore one critic. But a few hundred critics? Not so easy. It’s sort of like the “law of large numbers”.

I suggest that we all write letters, not form letters, but individual complaints, written in our own hand and in our own style. That won’t be so easy to ignore. Frankly I’m getting tired of the sloppiness, the faulty data analysis, and the arrogance we see displayed when obvious and simple errors are brought forth.

Steve M, If you do provide an address and name, I’ll write my first letter following further review of the paper.

I encourage others to do the same. cc: Mann’s department chair at Penn State too. Letter writing campaigns are an effective and time honored way of encouraging change. Let us not miss this opportunity. Complaining about the errors in the paper on this blog or others does nothing to further the quality of science, direct action is needed.

#17. I don’t think that this isn’t really an issue to take a stand on. But it is pretty funny. It was the gross carelessness of Mann that initially caught my eye in 2003. The refusal to correct known mistakes says quite a lot about Mann.

The location dots on the map projections used to show climate stations are not a good guide. Replot them on an actual globe and see the weighting towards high northern latitudes. Remember how Greenland shrinks on a spherical map. The look at the scarce data around Antarctica. It’s not a surprise that you can transpose intercontinental data and arrive at the
“highest skill” result possible.

“Skill” is a word to be avoided in a scientific publication. It is a human attribute, and a malleable one at that.

What would happen if buzz-words were outlawed, plain English was allowed and suppositions were supported by reasoned data?

I was amused by –

In each of the scenarios
explored, the reconstructions of NH mean temperature over
the reconstructed (A.D. 8501855) interval closely follow
the true histories, and the actual series lie within the
estimated uncertainties of the reconstructions.

I thought we were starting out to find the tue histories, not assuming that we knew the true histories and then inventing data to pad them out.

How much prior adjustment has gone into the figures used for examples? My mind is still stuck way back trying to understand the types of corrections for UHI, lapse, radius of influence, etc for surface temperatures. I just do not believe the most modern versions because I can see mistakes at the very start of the adjustment process carried through subsequent adjustments.

In an odd way, I’m glad that I’m old enough to die before science reaches its lowest levels of respect from the present trajectory. A life career doing it the right way, an end of life when the fundamental principles are treated by cowboys.

FWIW, I agree with Steve. This is not an issue to take a stand on, although it’s very amusing. I suspect enough people in the climate field follow CA for the point to be made.

I regularly chime in to say that it’s pointless to speculate about motives, but you have to wonder why Mann makes some of the choices he does, especially when he knows that Steve is looking over his shoulder.

“#20 #21 FWIW, I agree with Steve. This is not an issue to
take a stand on”

Even though Al Capone was guilt of numerous counts
of murder, grevious bodily harm and extortion, all
very serious offences, they finally did him for the
much less serious charge of fiddling him income tax
(admitedly a very big fiddle!).

Most sciences types are usually very meticulous and
careful (yeah, I can hear the rejoinders now) with
how they do things. This is what prompted JEG to
jump into the fray at what he regarded as a sloppy
piece of work by Loehle. Things like just sticking
the wrong data into your code, having it broadcast
in a public forum, does have an impact on ones
reputation. People then begin to wonder how much
of the other stuff that is published is also polluted
by errors.

Re #6, “I doubt any journal would consider publishing a comment
of that sort.” I disagree with this, I have published
comments of the type I am advocating. If there are
any published figures that depend on the data, then
these figures might have to be rerun and possibly
republished.

Besides, I reckon it would be really interesting to
see the correspondence that would be generated at the
journal when this particular turd gets thrown into
the swimming pool.

Surely the issues here are not really AGW, but a display of unprofessional behavior, an all too human flaw and the difficulty in getting bureaucratic entities to live up to their own codes of conduct, viz., Nature. It is hard for me to see this error or even Mann as pivotal in the entire AGW debate. To argue thus makes it too easy to dismiss the issue as the ravings of denialists. For me, AGW remains a possibility. The poor scientific practices, the professional hubris, and the CYA behavior of Journal Editors are facts.

I admit English grammar isn’t quite the same as science, but your statement:

I dont think that this isnt really an issue to take a stand on.

does not equate to the “I agree with Steve. This is not an issue to take a stand on” which James and some other’s as well as you apparently think. It’s the grammatical equilivent of an incorrect minus sign in an equation.

I have a hard time wrapping my mind around Mann’s mentality here. There were a couple of pretty easy exit strategies and it really says something about Mann that he didn’t avail himself of them.

Any businessman would have leaped at the chance to correct the location errors in the fine print of the 2004 Corrigendum SI even if he didn’t mention it in the Corrigendum itself. It was an absolute laydown – Mann could create a clean ongoing paper trail; the Nature editors had been totally co-opted so that he could say that the errors “didn’t matter”; there was no peer review of the Corrigendum and the Nature editors didn’t even read the Corrigendum SI. It was a completely free play.

Having missed that opportunity, then you have to ask yourself why he wouldn’t correct it in the 2007 SI. In the first paper, the error was undoubtedly just that: an error. But in the 2nd paper, it’s different. Mann knew of the error and nonetheless continued doing his calculations without correcting the error and publishing an SI with the known error still in place.

Did he think that no one notice? It’s hard to imagine that he thought that I wouldn’t at some point re-visit the points raised in MM2003 in the new SI.

Or maybe he thought that because he’d been able to tell the NAS panel that he hadn’t calculated a verification r2 statistic because it was a “foolish and incorrect thing to do” and no one had held him accountable, that he could avoid admitting the mis-location error even in the fine print?

Does he think that climate scientists are sheep? To which question, climate scientists have a one-word answer.

This is the third time that Ive seen a goofy collation error in MBH materials. In the original proxy data set that Rutherford directed me to, all the PC series were off one year. This was the famous data set that Mann subsequently said was the wrong data set and deleted, reproaching me for actually relying on the data set at his website to which Scott Rutherford had directed and which Id even asked Mann to confirm that this was the correct data set. (A new archive suddenly materialized in Nov 2003.)

A few comments:

IIRC, Mann was a new PhD when he published his first reconstruction, right? Just fresh from the defense period? I would suspect it’s quite rare that one’s first major published paper gets the kind of attention from larger science bodies that his did. I would suspect that for most scientists, their first published paper is like a rite of passage that sits in the shelves and may garner a few references by others in their field. MBH has been wildly successful if you count influence and attention from other scientists, politicians, the public, scientific bodies. That it had apparently not had its data checked very thoroughly seems to point to a gaping hole in the peer review process. I’m not excusing anything — it’s just that one wonders if these kinds of mistakes are a rookie thing or endemic.

Scientists are human. All the shenanigans and failings you find in your average person will be expected to crop up somewhere in the scientific community. It’s the same with physicians (with whom I currently am more familiar through my employment). You want your doctor to be a brillant scientist, diagnostician and a honest person, but sadly, they don’t always go together and you get an arrogant person with terrible diagnostic skills (how could they get through med school?) who damn near kills you.

That’s why I think this data quality “puzzle” solving is important. All kinds of interesting things will be discovered when there’s close oversight. You wouldn’t believe some of the stuff uncovered through the accountability processes . . . or maybe you would. :) I don’t want to say any more to protect myself, but suffice to say, errors abound. Why, I’ve even been known to transpose a line of data by accident myself, justlike in Mann. Luckily, my supervisor found it before the data went to the elected types. There is no reason to think science escapes sloppy work. Oversight is essential.

So, while this doesn’t surprise me, it does induce a very deep cringe.

Steve: Susann, I’ve got lots of experience in the world and know that errors occur and that’s why you have audits, engineering studies and independent due diligence. That a fresh PhD would do sloppy things doesn’t surprise me in the least and it’s unfortunate, but not particularly blameworthy on his part. However increasing blame does attach up the food chain and that’s probably why this story has hung around so long. I got interested in MBH when Mann said that he had “forgotten” where the data was, Scott Rutherford said that it wasn’t in any one place and that he’d assemble it for me, and I realized that no one had ever audited the work. At the time, I didn’t realize that academic work wasn’t audited in the way that would be done in business. So out of the blue I decided to verify the calculations, purely for my own amusement. I’d never published an academic article and had not expectation that anyone would be interested in the results. The response has really created much of the story and it amazes me that errors identified in 2003, even minor ones, are perpetuated in 2007 without any apology. Also blame increasingly attaches when the initial errors remain uncorrected and perpetuated. The repetition of the fresh PhD errors in a submission by the now seasoned author in a 2007 JGR article is different entirely.

#30. Now that I think of it, this article was published at JGR and it is a new rendering of the SI. So JGR has its own obligations in the matter and Nature’s failure to correct the error doesn’t mean that JGR won’t require the error to be corrected. In the case of Nature, the correspondence was amusing. That was pre-blog days and before I had an audience. If the issues in our Materials Complaint had been played out in the full sunshine of the present audience, they’d have had to do things differently.

So it becomes much more of a hot potato for JGR than it was for Nature. It’s also much worse for them because the errors had already been identified and the authors persisted in covering them up.

The only practical difficulty is that, if I open the file, this isn’t going to be the only item on the agenda.

Stan, no, teleconnections you know. The Mannomatic does not care about the location. That’s the “beauty” of teleconnections, and for instance Mann is using SH and precipitation proxies for his NH reconstruction, I find it rather hypocritical of JEG and others to critisize Loehle for not discussing spatial coverage while at the same time ignoring the fact that reconstructions relying on teleconnections and global temperature regression do not even have a way of guranteeing any adequate spatial distribution of the proxy weights.

It was the gross carelessness of Mann that initially caught my eye in 2003. The refusal to correct known mistakes says quite a lot about Mann.

Couldn’t agree more! This is what will always make me also highly suspicious of: (1) any research in which Mann participates and (2) any person who defends or talks circles around Mann’s errors.

In my experience (somewhat similar to Steve Mc’s out in the business world), hand-waving your way past obvious errors may work once or twice, but eventually you’ll get fired when the right people realize your work is crap. And I’ve never seen my colleagues rally around defending an obvious mistake. In fact, they’re much more likely to throw the mistaken party under the bus.

The most successful people I’ve been around in my career have insisted on quality in everything they do. It’s a mindset that builds on itself – quality in the little things begets quality in the big things. Similarly, little sloppiness breeds big sloppiness.

SteveM, is it possible that the table is wrong, but the input is correct?

That is he read the data in correctly and processed correctly, but when he wrote the lat lon
out in table form he made a mistake at that stage?
Steve: Yes, he made a mistake in the lat-longs – isn’t that exactly what I said? The Paris precipitation series is the “right” series, it’s just in the wrong location. I don’t know where some of the series actually come from, but they look like real precipitation series; perhaps they are combinations of a few nearby stations. The issue is the geographic mislocation, not the transcription of the series.

@Susan–
I’m tempted to just say “what Susann said”…. but there are ohhh so many more things that could be said. For example, this incident highlights how the ordinary scientific peer review process breaks down when a subject suddenly has trememdous policy implications. For better or worse, the ordinary scientific peer review process is designed to tolerate and identify error without necessarily causing severe loss of face on the part of individual researchers who introduce novel ideas, new analysis techniques, or who may simply fail to catch their own goofy mistakes.

In the ordinary course of events, only researchers would have read this paper in detail after publication. Those in dendrochronology applied to climate change would have discussed it in detail with Mann. Problems of moment would have been discovered by those who tried to replicate it.

Collocation errors might be found– but no one would write Nature. If they did, Nature would not report that as news.
What would happen instead? If the problems were major, researchers would simply drop the idea of using Mann’s techniques. If the problems could be rectified, papers would cite Mann, and point out it’s difficulties and flaws. The reason to point them out? Correcting them to obtain a realistic result would be the contribution of the new paper.

All of this would have been fairly quiet, and not particularly embarrassing to Mann.

But what happens when the results of a paper almost instantly become the poster child of a political group? When it’s results are used to support decisions to fund major research programs? When finding a different result is upsetting to major programs? Should Nature change it’s policy about not reporting clear, goofy errors that might seriously impact the results of a paper published years ago?

To an extent, I’m not surprised Nature didn’t– because that’s not what peer reviewed periodicals normally do. They normally just let the process of ignoring mistakes work!

It is all rather sad when politics overtakes science in this way.

It’s also why more contentious engineering research has two sets of reviews. The peer review process operates in scholarly circles. A entirely different review process sifts for other types of errors– including bone headed ones. Climate scientists aren’t used to the type of scrutiny and review that occurs in Nuclear Regulatory circles, waste clean up issues, weapons development, or even just engineering safety issues are used to dealling with.

But the fact is, the NUREG, DOE, EPA type reviews are important when policies are going to be implemented, or things are going to be built. This type of review is required if results climate science are going to drive policies.

Lucia, #43, worth saying twice. Why isn’t the second layer of review in this policy matter? More money and social dislocation hangs on this than on all those other things which get double reviewed.
======================================================

#45. The location errors don’t make a speck of difference to the final NH reconstruction, which is one of the many reasons why it’s crazy that Mann didn’t correct the errors. The actual location of any Mannian data doesn’t matter for this calculation – however they do affect the table of correlations in the new SI, and all the correlations involving the precipitation series are wrong.

For the final recon, all the series could have been placed within 10 meters of the South Pole and the answer would have been the same. You could replace nearly all the proxies by white noise, as I’ve done in some experiments, and it doesn’t affect the results as long as you have a couple of HS-shaped series (which are contributed by the Graybill bristlecone chronologies.) But try to take out the bristlecone chronologies and they squeal like stuck pigs.

BTW did I mention that word “bristlecone” is not mentioned once in Mann et al 2007 even though the bristlecones are used large as life, and even the incorrect Mann PC1?

Collocation errors might be found but no one would write Nature. If they did, Nature would not report that as news.
What would happen instead? If the problems were major, researchers would simply drop the idea of using Manns techniques. If the problems could be rectified, papers would cite Mann, and point out its difficulties and flaws. The reason to point them out? Correcting them to obtain a realistic result would be the contribution of the new paper.

Hmm, so maybe it’s better to “bend” Mann than to “break” him. i.e. Somebody writes an article with the purpose of “extending” Mann’s results and mentions his errors in the process and obtains a realistic result. Where realistic could mean contradictory or inconclusive.

@Kim,
I suspect the reason for lack of a second review policy is historical.

Until recently, climate science was a purely scientific matter. Climate scientists don’t build anything. If the models are flawed, nothing blows up and no one dies etc.

At DOE, the rules for verification and validation of software formally account for consequences of a mistake. So, you can imagine that a code controlling a robot that might be used to hold a camera while cleaning out 1 million gallon tank of radioacitve waste would come under close scrutiny. After all, what happens if, due to a code, theory or design error, the robotic arm travels too far and punches a hole in the side of the tank?

Similar questions get asked when running thermal-hydraulics codes. Agencies do not rely on scholarly peer review for these things. The idea it would work is laughable.

But consider climate change as a scientific endeavor back in 1980. What was the consequence if a code contained a mistake that resulted in more or less incorrect results? The worst that would happen is a poor paper might get published; maybe there would be a bit of confusion among specialists for a while. Why fund a large formal oversight committee for that?

Now move to 1990. We start to see more formal large projects. But the consequences are still not “people will die if the data aren’t reviewed in detail”.

So, in the past, these formal reviews common to the EPA, Engineering, FDA etc. weren’t required in climate science. Instituting them would have been costly, and actually impeded scientific process. (When politics and policy aren’t involved, ignoring papers based on mistakes works really well over time.) But, now the more formal reviews likely are required.

Unfortunately, this should have been forseen around the ’90s, but it wasn’t. So, here we are with the mistakes being found by … a retired mining engineer. (Do I have that right?)

@Patrick M:
Bending instead of breaking is precisely what happens during the ordinary peer process. However, for this to work, things need to be more private than possible when the science suddenly becomes very important to policy decisions.

I sort of feel sorry for Mann.. but that doesn’t mean he’s justified in refusing to reveal stuff that a) he would have revealed more privately in the ordinary course of events and b) is important information on which others — including voters who include high school drop out hamburger flippers– form opinions on policy decisions.

I am unclear on something – how do precipitation series fit in to Mann’s method. I thought he used tree rings and borehole cores. Is precipitation also considered a proxy for temperature??
Steve: In MBH98 and now in Mann et al 2007, instrumental precipitation is used as a proxy. JEG argues that precipitation in one place teleconnects to temperature in another and that such teleconnections will be revealed by Mannian algorithms. In this respect, it’s interesting to read Mann’s excoriation of Soon and Baliunas and then try to figure out how Mann’s use of precipitation series avoids the error that attributes to others.

November 22nd, 2007 at 8:51 am
#45. The location errors dont make a speck of difference to the result, which is one of the many reasons why its crazy that Mann didnt correct the errors. The actual location of any Mannian data doesnt matter. All the series could have been placed within 10 meters of the South Pole and the answer would have been the same. You could replace nearly all the proxies by white noise, as Ive done in some experiments, and it doesnt affect the results as long as you have a couple of HS-shaped series (which are contributed by the Graybill bristlecone chronologies.) But try to take out the bristlecone chronologies and they squeal like stuck pigs.

BTW did I mention that bristlecones are not mentioned once in Mann et al 2007.

Then which proxy gave us a HS in this report?

Steve: The bristlecones and the incorrect PC1 are in the data large as life. It’s just that the word “bristlecone” is not discussed and the impact of Graybill bristlecone chronologies not analyzed.

Susann, it’s true that a fresh PhD doesn’t normally expect the kind of promotion and attention that Mann got, but don’t forget, he was a Lead Author of the IPCC report (Ch 2) that promoted his result so heavily. If he had wanted to put in some cautionary language, or tone down the presentation, he could have, and I believe other chapter authors had counseled him to do so.

Anyway, nice work Steve. It’s easy to get bogged down hunting for some big complex explanation for data discrepancies like the precipitation series. It takes a sharp eye to spot a simple explanation. I suppose, too, that it helps if you’re immune to the assumption that people this smart can’t futz up their tables like this (even after supposedly publishing a Corrigendum).

Steve M, now I’m confused. My comment in 18 was based on what was in your original post:

But because Mann has not corrected the geographical locations of his precipitation series, all the calculations showing correlations to the gridcell location for these series are incorrect – despite this matter being brought to his attention long ago.

but now you’ve written in 47:

The location errors dont make a speck of difference to the result, which is one of the many reasons why its crazy that Mann didnt correct the errors.

Which is it? Do the lat/lon errors make a difference in the result, or do they not?

If they don’t, I can now understand your reluctance to take a stand on the error, which previuosly made no sense to me as to why you would not want to. Please clarify.

Steve: They affect the table of correlations in the new SI where Mann compares Paris precipitation to temperature in another gridcell – so the relevant calculations in the SI table are wrong and the map showing a data point in Bombay is wrong. In 47, it would have been clearer if I’d distinguished between the correlation table and the NH reconstruction: it doesn’t affect the final NH calculation which doesn’t use the geographical location. So it depends what “result” you’re talking about. I’ll edit #47 to clarify.

RE 50. What lucia said. A bunch of people here have worked in aerospace. Some have seen
friends die in machines they built. These people take auditing of code and data seriously.
They get a bit tweaked when they see guys who want to “fly the planet” slough off mistakes.
Makes them remember the fireballs and the brave men who rode their mistakes into oblivion.

I think it’s more accurate to say that it is scientists who have taken over politics; and it’s the politicians who have allowed this to occur. In a society, bad things happen when unelected, unaccountable people start making national (and trans-national) policy.

My opinion is that Mann doesn’t make the changes because that would mean he recognized the existence of Steve McIntyre. I know it seems crazy that he refuses to deal with Steve but I think that in this respect he really is crazy. As long as he can claim that Steve is a nonentity and has been competely eliminated from consideration, he doesn’t have to face the reality that he’s made mistakes. That’s the reason for no link from RealClimate to ClimateAudit, no mention in articles, no willingness to meet him even in a Congressional Committee, etc.

And since Steve is the one who pointed out the mistaken locations in the table, he can’t correct them.

The location errors dont make a speck of difference to the result, which is one of the many reasons why its crazy that Mann didnt correct the errors. The actual location of any Mannian data doesnt matter. All the series could have been placed within 10 meters of the South Pole and the answer would have been the same. You could replace nearly all the proxies by white noise, as Ive done in some experiments, and it doesnt affect the results as long as you have a couple of HS-shaped series (which are contributed by the Graybill bristlecone chronologies.) But try to take out the bristlecone chronologies and they squeal like stuck pigs.

One of the things that really bothers me about the paper is the title. Yes, you always get a hockey stick if you use bristlecone pines (or a PC derivative proxy such as Mann’s PC1) along with a variety of inappropriate statistical methods. But that’s not the same as showing that the hockey stick is robust to a wide variety of proxy combinations, both including and excluding BCPs.

Here is the money quote from Mann, et al 2007 (emphasis mine):

Nearly a decade later, more than a dozen studies using alternative proxy data and reconstruction methods have, moreover, independently reaffirmed earlier studies such as MBH98, producing millennial or longer hemispheric temperature reconstructions which agree with the those reconstructions within estimated uncertainties. These additional studies support the key conclusion that late 20th century/early 21st century warmth is anomalous not only in the context of the past millennium, but apparently at least the past 1.5 or 2 millennia.

How can MBH98 or 99 be independently reaffirmed when the studies use PC1?

Also, those of you who believe that it is unconstructive to be “obsessed” with the hockey stick, read the last sentence. From a policy-makers perspective, that is the kind of definitive statement that moves poll numbers and gets votes.

IIRC, Mann was a new PhD when he published his first reconstruction, right? Just fresh from the defense period? … That it had apparently not had its data checked very thoroughly seems to point to a gaping hole in the peer review process. Im not excusing anything  its just that one wonders if these kinds of mistakes are a rookie thing or endemic.

Which kind of puts the ball back with the senior co-authors, n’est-ce pas? (as they say in New England)

The longitudes are inconsistently reported – sometimes the longitudes run east, sometimes west and sometimes around the world.

The first 80 series have a Greenwich meridian with positive longitudes to the east, negative to the west. The ITRDB North America PCs have the same orientation but the locations are denominated as +260 etc. It would be interesting to know how Mann chose a gridcell for each of the NOAMER tree ring PCs.

The rest of the series have a reversed orientation: west is shown as positive and east as negative. It’s an odd way to arrange a table. You’d think that he’d have sorted out a consistent orientation for east and west before publishing the table.

Here are a couple of thoughts on Mann’s 2007 paper.
First, on page 15, they say:

This result (as well as the separate study by Wahl and Ammann [2007]) thus refutes the previously made claim by MM05 that the features of the MBH98 reconstruction are somehow an artifact arising from the use of PC summaries to represent proxy data.

Well, if we had argued that the only problem in MBH98/99 is the PC error, and that the PC error alone produces the MBH hockey stick, then this paper and its triumphant conclusion might count for something. But we argued something a tiny bit more complex (though not much). The PC’s were done wrong, and this had 2 effects. (1) it overstated the importance of the bristlecones in the NOAMER network, justifying keeping them in even though they’re controversial for the purpose and their exclusion overturned the results. (2) It overstated the explanatory power of the model when checked against a null RE score (RE=0), since red noise fed into Mann’s PC algorithm yielded a much higher null value (RE>0.5) due to the fact that the erroneous PC algorithm ‘bends’ the PC1 to fit the temperature data. Mann’s new paper doesn’t mention the reliance on bristlecones. Nobody questions that you can get hockey sticks even if you fix the PC algorithm, as long as you keep the bristlecones. But if you leave them out, you don’t get a hockey stick, no matter what method you use. Nor, as far as I can tell, does this paper argue that MBH98 actually is significant, and certainly the Wahl&Ammann recalculations (http://www.climateaudit.org/?p=564) should put that hope to rest.

Second, maybe I’m missing a nuance, but the diatribe against r2 (and by extension, Steve et moi, para 64) is misguided on 2 counts. They say that it doesn’t “reward” predicting out of sample changes in mean and variance (paragraph 39). Where I work, we don’t talk about test statistics “rewarding” estimations, instead we talk about them “penalizing” specific failures. The RE and CE scores penalize failures that r2 ignores. We argued that r2 should be viewed as a minimum test, not the only test. You can get a good r2 score but fail the RE and CE tests, and as the NRC concluded, this means your model is unreliable. But if you fail the r2 test, and you “pass” the RE test, that suggests you’ve got a specification that artificially imposes some structure on the out of sample portion that conveniently follows the target data, even though the model has no real explanatory power. Another thing that’s misguided is their use of the term “nonstationary”. They say that RE is much better because it takes account of the nonstationarity of the data. Again, where I work, if you told a group of econometricians you have nonstationary data, then proceeded to regress the series on each other in levels (rather than first differences) and brag about your narrow confidence intervals, nobody would stick around for the remainder of your talk. And you’d hear very loud laughter as soon as the elevator doors closed up the hall.

Third, what’s missing here is any serious thought about the statistical modeling. I get the idea that they came across an algorithm used to infill missing data, and somebody thought–whoah, that could be used for the proxy reconstruction problem–and thus was born regEM. Chances are (just a guess on my part) people developing computer algorithms to fill in random holes in data matrices weren’t thinking about tree rings and climate when they developed the recursive data algorithm. You need to be careful when applying an algorithm developed for problem A to a totally different problem B, that the special features of B don’t affect how you interpret the output of the algorithm.

For example, in the case of proxies and temperature, it is obvious that there is a direction of causality: tree growth doesn’t drive the climate. In statistical modeling, the distinction between exogenous and endogenous variables matters acutely. If the model fails to keep the two apart, such that endogenous variables appear directly or indirectly the right-hand side of the equals sign, you violate the assumptions on which the identification of structural parameters and the distribution of the test statistics are derived. Among the specification errors in statistical modeling, failure to handle endogeneity bias is among the worst because it leads to both bias and inconsistency. A comment like (para 13) “An important feature of RegEM in the context of proxy-based CFR is that variance estimates are derived in addition to expected values.” would raise alarm bells in econometrics. It sounds like that guy in Spinal Tap who thinks his amp is better than the others because the dial on his goes up to 11. So, the stats package spits out a column of numbers called “variances.” My new program is better than the old one because the dial goes up to “variances”.

It’s just a formula computer program, and it won’t tell you if those numbers are gibberish in the context of your model and data (as they would be if you have nonstationary data). You need a statistical model to interpret the numbers and show to what extent the moments and test statistics approximate the scores you are interested in.

Some people can’t admit they made a mistake, no matter how trivial. I don’t know if it’s hubris or what. Mann seems like one of those folks.

If Steve M made a tabular error on this blog, it would never be forgotten. If he made a tabular error in a publication – particularly one he had been notified years earlier to be incorrect – and used it for a correlation comparison, it would immediatley be used forevermore in an attempt and discredit him.

I was going to write something similar myself, but you’ve beaten me to it: Mann cannot face the idea that Steve can be right about something. I’d put a slightly different spin on the reasons though, I just think Mann lacks the maturity to accept criticism in the direct form it is being given. I suppose when you spend your life surrounded by yes-men, you don’t have the experience to deal with it.

Since the locations don’t affect the reconstruction, I don’t think there is much merit about creating a storm in a teacup about it. It certainly isn’t going to make Mann behave more like a fully grown scientist, if anything it will just entrench the two viewpoints to a greater degree than the situation we have today. Better to focus on the strongest arguments and main issues in this case.

#67. This is obviously old stuff. The key issues re Mann are bristlecones, spurious regression and overfitting, meaningless RE statistics and poor modeling, plus the incorrect PC method that’s been much discussed. Ross’ post is excellent.

I was reminded of this point because of JEG’s teleconnection argument. I’m sure that JEG’s response will be that a Mannian teleconnection between the Indian monsoon and precipitation in West Chester, Pennsylvania is just as meaningful as one to precipitation in Bombay, but most non-specialists will not be impressed.

The tabular error is a very simple one and in itself, shouldn’t be fatal. However, the continued use of a statistical method that was criticized by the NAS and a tree ring series in which questions exist about its value as a proxy of temperature seems very questionable, to put it in a polite manner. I haven’t read the article, but I would expect that there should be a very thorough explanation and justification as to why they are retained and why the method is retained, given past criticism. I don’t honestly have a good enough grasp on this to judge but given what I have read, I am perplexed.

Lucia, as ou say, perhaps if this wasn’t such a hot political issue, the usual processes of replication would have weeded this error out and the science would have moved forward without a lot of personal face-losing and stonewalling. Given the way MBH was used in policy documents like the IPCC report, that didn’t happen and the consequeces are a black eye for paleoclimate reconstructions — and the whole peer review process. It casts doubt on the science and acts as a threat to development of sound policy.

@Susann– I think Mann’s errors, by themselves, wouldn’t give the scholarly peer review process a black eye. What is giving it a black eye are some who want to claim the peer review process does something it is not designed to do: detect errors, inaccuracies and distortions and publicize the fact these were found in a manner that is both timely and clear. Scholarly peer review is not designed to do either of those things.

In fact, if scholarly peer review embrace the task of announcing errors in a quickly in a clear, loud voice, the system would inhibit anyone from advancing new ideas or techniques.

What gives scholarly peer review a black eye? Scholars coming forward and suggesting some analysis is right or convincing because it passed scholarly peer review. Scholars who suggest outsiders can’t see or comment on sometimes obvious errors, because the outsiders aren’t in the field (even when the errors involve the clear misuse of techniques widely used in other practices.) Scholars who obfuscate when asked questions about peer review.

Peer review plays a valuable role in science and scholarship. It hardly guarantees the accuracy of the contents of any paper even in the most prestigious journal.

#73. The issue is not about academic peer review and I agree entirely with Lucia. Journal peer review is what it is and the reviewers do a very limited (but valuable) gatekeeping function. One of the reasons why availability of code is a good idea is precisely because academic peer review is so limited. If some subsequent researcher wishes to do more detailed due diligence, then the difficulty of doing so is reduced by having data and code readily available. Mann’s record, at this point, is by no means particularly bad within the field. Lonnie Thompson is incomparably worse.

Lucia and Steve M — thanks for pointing that out. I suppose it’s when other scientists try to replicate the findings that any underlying problems become clear – in the normal, non-politicied, process. It’s that replicability that is so key and replication can’t occur unless the data is available and the methods known. It seems ill-advised that the IPCC featured a rookie paper using new methods whose findings hadn’t been replicated. This is all very educational, and so I thank you.

#73. This was a point that North made clearly at the NAS panel press conference – papers need seasoning.

The other practical issue for policy makers is that IPCC review authors are typically people with a horse in the race so to speak. Briffa was the author of the AR4 millennial paleoclimate section and he’s right in the thick of the battle. In business, you need an independent qualifying report and this doesn’t happen in IPCC. The rationale is that they are getting authoritative authors and they are; but there’s a real benefit to an independent report, as has been proved over many years in business prospectuses. The specialists need to cooperate with the independent review. Von Storch agrees with this point BTW.

In Mann’s case, he was also appointed IPCC AR3 lead author for the section containing the HS and needless to say promoted his own work, including an untrue claim about its statistical skill that an independent author might not have made. I don’t know the extent to which Mann knew about the heavy use made by Houghton and IPCC leadership of the HS (which had center stage in the AR3 press conference) but I suspect that he wouldn’t have resisted it.

So there was negligible oversight or review all the way to governments. Everyone assumed that someone had done due diligence and no one had.

I hope by now you have come to understand that in Mann’s view MBH98 is not ancient history, but rather the un-dead study that must live again. Monty Python does it best, but for this parrot Steve is the only man for the job. Steve continues to attack MBH98 only because Mann refuses to give it a proper burial.

It is unfortunate that JEG has throw himself into this fray, but maybe he can learn from the experience – adventures are never easy, but sometimes survivable. Given JEG’s challenge on Mann’07, I believe we are about to witness a rite of passage. Not all pass, but we can hope for the best.

Susann,
Do you see now why it is so ironic for the team to say they’ve “moved on” (meaning that M&M criticisms are no longer relevant because “the science has advanced”)? They haven’t “moved on”! The proxy recons are not at all independent, because the hockey team is addicted to bristlecone pines, the common element in all these recons. Not only that, but like any addict, they are always trying to hide their addiction, denying they have a problem. So it’s not just the science that’s questionable (choice of proxies, calculation of error bars), it’s the poor scientific practice, the lack of documentation and due diligence, the obfuscation. Hence the need to audit these studies.

I hope by now you have come to understand that in Manns view MBH98 is not ancient history, but rather the un-dead study that must live again.

Yes, I’ve learned my lesson. :) I have this image of a zombie rising from the grave, over and over . . .

Seriously, I am intrigued by this persistence. I can’t help but wonder why the tree ring proxies are kept in. Do Mann or others offer any explanation why they should stay in even in light of the NAS panel comments?

I hope by now you have come to understand that in Manns view MBH98 is not ancient history, but rather the un-dead study that must live again. Monty Python does it best, but for this parrot Steve is the only man for the job. Steve continues to attack MBH98 only because Mann refuses to give it a proper burial.

I wouldn’t presume to speak for Steve, but it’s hard to think of a scientific paper that has been as comprehensively demolished as MBH9x. The main reason it keeps coming up is that it is so entertaining (hence my remark a while back “MBH – the gift that keeps on giving”). Mann might maintain that MBH9x is all tickety-boo, but it’s hard to believe that many in the climate community entertain this possibility, even if they don’t say so. If I’m wrong about that, then the state of climate science is worse than I imagine.

The more important issue is that problems in MBH keep getting reticulated into more recent reconstructions, especially the BCPs and Mann’s PC1. When these papers are cited by the IPCC, it becomes less entertaining and more serious, given that the evidence is relied on by policy makers.

FTM, do you really think the fate of AGW rests on the status of MBH and BCP? AGW seems to have a great deal more political traction that generated by Mann and his paleoclimate proxy reconstructions. There are still so many complexities to be explained and understood in climate science, from what little I’ve read of it, and there are a great deal of other supports for the notion of AGW yet to be put in the grave.

Re #82
There are a number of warmer-now biases:
-in the proxy recons (means and errors)
-in the underestimated UHI effect
-in the back-inferencing step by which unexplained warmth predicted by GCMs is attributed to GHGs
-increased detection of early-stage hurricanes

It behooves a scientist to ask what would be the net outcome if all these warm biases were removed.

The Mann et al un-dead are simply the product of the unholy union of politics and science. Science gratefully accepts criticism and moves on, denying criticism is the coin of politics. Giving in on the Bristle Cone tree rings would involve losing face. Its not science, but it is real coin. Throw out the Bristle Cones and you throw out the hockey stick.

The major point of Climate Audit is to get climate science back into the realm of science so that the climate issues can be properly dealt with. Just the facts Mam, that’s all we want. One would hope that is what the policy makers want as well.

#81. No, the main AGW arguments lie elsewhere. In a way, MBH has been used for promotional purposes and the “real” arguments lie elsewhere. But it’s really really hard to find a clear articulation of how increased CO2 leads to 2.5 deg C in a clear way. That doesn’t mean that the relationship doesn’t exist, only that no one has provided a reference to a clear explanation despite many months of requests.

#79. James, actually Wahl and Ammann, in its own way, is as entertaining as MBH. You have to get past even more orotund verbiage but it has taken spurious regression to an absurd extreme.

Lest you or others mistake me, when I speak of politics I am not talking about left/right, but rather the human condition. The fast track in any human group is politics. Government, Academia, Business, Gardening Club, it is all the same. The choice is merit or politics. Not that politics is without merit, but it is of a different order.

Not long after Hansen released his code after pressure from Steve, I warned about simple errors such as some vagaries of spherical geometry when close to a Pole and others. So here we have longitudes given wrong signs, lines partially dropped from tables, precipitaion figures from wrong Continents and so on as described above.

These things are NOT common in the better scientific world. When you are involved in calculating the maximum radon dose that uranium miners should receive each year, or die, you do not let simple errrors like this slip through. You have levels of confirmation and checking to minimise that possibility.

Lucia is right when she notes that newly-emergent, Earth-shattering results distort the confirmation process. Like the “Global warming will kill us all” theme. The matters that start Lemming runs. I lived through the catch-cries and dogma of the anti-nuclear movement. I must have written several hundred times that the standard of “green” science, like the USA EPA started, was woeful. Why do you think we still have mantras that “Used reactor fuel has to be managed for 250,000 years” because a pseudo-scientist looked up that Pu-239 had a half-life of 25,000 years? Why do we persist with the dumb idea that “harm results from all doses down to zero”?

Now we have an abundance of carbon trading schemes that will grow more trees, forgetting that the trees will eventually decay to produce CO2 in the air (unless, unlikely, they form coal). The science is wrong, wrong, wrong. The snake oil salesmen persist, under the mantle of their idea of science.

Back to the old quote that says approximately that evil can flourish simply by the uwillingness of one good man to stand up and be counted. Good man, Steve.

#87 – Steve M, I’m still at the early stage of comprehension of the wherefores and whys of MBH and MM — i.e. statistics and BCP/proxy use. I appreciate the short time I’ve spent here, for I have learned a great deal. Or at least, more than I understood before. In my understanding, we know there is a climate history and that this climate history may have stories to tell about our current climate and its apparent warming. Our ability to capture it using science and discoer its lessons is in question. We have proxies, but the value of each proxy is the subject of debate, some more than others. Using the proxies to recreate the climate history involves a lot of assumptions about the proxies and what they say and how valid each one is. The means to combining them to show a global climate history record an then interpreting what that recreated record says about today is even more contentious, as evidenced by the MBH zombie. It seems that Mann and others feel the zombie is not quite dead, the disease misdiagnosed, and deserves to be brought back to life. That thinking is what I am next going to explore, if I can find the material or ask the right questions.

The 2.5C question is a ways off and I understand, not the subject of this blog. When I feel I’ve wrestled this dragon a bit more, I’ll see what I can find on the 2.5C issue.

RE 92. Since he just lost a fool’s mate game by suggesting a reading of Mann2007, and since his
students have obviously read this, He is probably going to claim a mere flesh wound, and want to
call it a draw

@steven mosher–
You know what might be interesting? If JEG, Judy or one of the GA tech students took a video recorder to class and posted the class lecture on Mann 2007 on youtube. Then we could all see the range of discussion and views.

I’d like to get up to speed on this, and I think that would be rather interesting. The technology exists. It would be kind of cool!

RE 94. Interesting indeedy. Long ago I used to tape lectures. One for posterity ( the famous guys)
two, because I got sick and tired of frat brothers asking for my notes, three because I made money
by transcribing notes and selling them. I got paid good money.

I suspect not much as changed in the university. You get one kid who challenges what the professor says.
95% of the time the kid is a crank…the other 5% of the time….

@steven– I wasn’t thinking of taping the lectures as a means of challenging the professor. I was thinking more that since JEG has placed discussing climate audit and Mann’s work on the class syllabus, and JEG stopped by here, it would be interesting to see the class. It might also save him time, since many here asked his opinion on the Mann paper.

It’s technologically easy to make tapes. So, seeing a tape might be interesting for us and the discussion JEG started here. We would see if and how he addresses issues like simple sloppiness, R2, teleconnections etc

I read up a bit and I know understand what the word teleconnections means.

I appreciate that the BCP “secret sauce” is the real issue, and has been since MBH98, but the tabular screwups — and Mann’s refusal to fix them, a decade later — are too rich, too absurd, too… Monty Python to let pass. Mann is still spinning his pseudoscientific blather, his Team is still behind him, 95% of even scientists can’t follow the statistical arguments….

But everybody understands hubris, and idiotic stubbornness, and refusal to face critics. It makes a great story, that anyone can follow, and opens the door, for anyone not totally close-minded, to all the other good arguments made here about sloppy Climate Science and spending terabucks on stuff that may well do more harm than good. It would be a real pity to bring on the (overdue) next Ice Age, while impoverishing ourselves in the process.

Hmm. So what’s needed, besides you guys writing up a formal technical note to JGR? (which absolutely needs a weblink back to the “rain in Maine… on the Seine” stuff here.)

We need a popular treatment of this, in an influential publication. Hmm-2. The WSJ is already sympathetic, and has a history of running long debunking articles. Any readers have contacts there? Any hungry pop-science writers in the audience?

Which brings me to the next point: I (and many others) think CA needs a better “Introduction for Newbies” section, partic since traffic is up with your major award. Steve, you don’t need to spend your limited time on this — this is where guys like me, who can follow your arguments but don’t have the technical background to advance them, could make a real contribution. A wiki-style collaborative effort would seem the way to go. Steve, could you start a new thread for this, so we can get the ball rolling?

Teleconnections. The simplest explanation I got was that weather in once place was connected
( correlated) to weather in another place. The physical mechanism was rossby waves.. Stationary
patterns in circulation. My sense was that the teleconnection argument was merely a way of hand
waving away sparse sampling. That is, I sample battle creek michigan, it’s correlated with Paris,
therefore I’ve “effectively” sampled Paris.

Bender, however, has explained it quite differently. So… Time for me to visit primary texts.

Re #100 – **Which brings me to the next point: I (and many others) think CA needs a better Introduction for Newbies section, partic since traffic is up with your major award. Steve, you dont need to spend your limited time on this  this is where guys like me, who can follow your arguments but dont have the technical background to advance them, could make a real contribution. A wiki-style collaborative effort would seem the way to go. Steve, could you start a new thread for this, so we can get the ball rolling?**
I was thinking about this the last couple of days. See Unthreaded.

@Steven mosher–
Well… if two locations temperatures ‘X’ and ‘Y’ are correlated, then yes, you get some predictive information about ‘X’ when sampling ‘Y’. In the stock market, you get some information about what happened to tech stocks by knowing what happened to one tech stock. You get some information about what happened to Japanese stocks by knowing what happened to one Japanese stock.

I am under the impression that the spacial cross correlations in climate aren’t monotonically decreasing functions with distance and this is due to some structure in atmospheric and oceanic flow pattern. And that seems to be all “teleconnections” means.

I think in portfolio theory, these sorts of ideas about cross-correlations are used to figure out how to get the broadest degree of diversification while buying the fewest numbers of different types of stocks. (Because of structure and known patterns of correlation, you can get decent diversification by not selecting stocks entirely at random.)

So, presumably, you can do similar things when trying to estimate global temperature. It strikes me it would take rather a lot of care to do it correctly. But, yes, in principle, a thermometer in CA on place might, statistically speaking, give us a guess about what’s happening in Paris. (Though, I bet Parisian’s rely on thermometer in Paris.)