Thursday, November 26, 2009

Yet More on the CRU "Hack"

* Gavin Schmidt (climate modeler for NASA) has done a good job providing his version of events, and he tries to defuse the most "shocking" emails here.

* CEI is threatening a lawsuit if Schmidt's employer (NASA's Goddard Institute for Space Studies, or GISS) doesn't comply with some FOIA requests. The one part I found a bit much was where they seem to be threatening to sue because Schmidt wasn't fair in his comment moderation at RealClimate.org during work hours--and hence he was impeding science "on taxpayer time." C'mon.

* I originally did not refer to this incident as a "hack" because I thought it was an inside job. For one thing, that was the first report I read. But I also thought that an outside hacker wouldn't have known where to look to grab all this stuff. An anonymous poster at a previous Free Advice thread sums up my view well:

If this is the result of a hack we are dealing with the best hacker the world has ever known hands down. When a hacker enter a system, they have a short amount of time to rake in whatever they can find and then GTFO! This is a very comfortable package that has taken a conciderable amount of time to assemble and not something that was done in the span of a hack. Then the hacker should have worked for months, infiltrating, analyzing and collecting. A server is a HUGE filesystem. Those who believe that this is the result of a hack, will have to similarly believe that you can indeed find not 1, not 2, not 10 but 100+ needles in a haystack the size of Kansas. Either the "hacker" knew exactly what they came for and was able to navigate through the filesystems and find all the relevant individual parts which points towards an insider, or the hacker new exactly what package to get. This itself raises questions. who then packed this package and from where did the hacker know what to look for and where? The last suggestion is pure luck, but then again...who compiled the package of data? I think it was an insider job and like many others I think it may be "Harry" who either did it, or compiled the package and then told someone where to look. No evidence for this, but the sheer amount of luck required by a random hacker to come upon this package is just unacceptable.

Now I don't really know enough about computers to say whether the above is true, but that's where I was coming from when I initially attributed this to a whistleblower. Or are we saying that there was so much skullduggery going on at CRU, that a randomly grabbed chunk of emails contained a handful of zingers?

CRU, in my opinion, is effectively finished. Who would believe them or use their info now? The damage they created, exposed by the 'CRU Hack', will keep their movement pinned down for some time to come. I agree that it is an inside job--of the Pentagon Papers level.

Since I've been following the CRU story rather closely, I feel obliged to comment on Gavin Schmidts so-called "explanations" :

About Kevin Tranbearth "he is concerned about our inability currently to track small year-to-year variations in the radiative fluxes".

This in response to Kevin Tranbearth saying that "We cannot account for the recent lack of warming, and it's a travesty that we can't." It doesn't sound like he is "concerned", it sounds like he knows that the models are flawed.

The issue with sea-temperatures in the 1940's : "The impact of this has not yet been fully corrected for in the HadSST data set, but people still want to assess what impact it might have on any work that used the original data."

This is the explanation to an email where the discussion is regarding how to make the sea-temperature blip in the 1940's "disappear" with a suggested manipulation that would offset it by 0.15C.

The Harry_Read_Me : This file cannot be explained away. It clearly shows that the database at CRU was in a complete, utter mess, no one knew which data belonged where, some was missing, some was treated differently than other. I can also recommend the GRIDDING_README.txt where someone tries to make sure that people know that the cloud data from the past century is now lacking original (unadjusted) data, because someone LOST IT. There are also code that suggest that datasets have been loaded, manipulated and then saved back to produce new "adjusted" data-sets which contains suspicious hockey-stick shapes.

The "decline that was hidden" is described at CA and WUWT. Several reviewers were against the publishing of the referenced graphs that had their "declines hidden" because it was misinformative.

Basically, RealClimate has started a campaign where they reply to all issues with non-answers that touch on the subject, but are in fact answers to a different question that was asked.

But don't take my word for it - browse the files yourselves. If they don't convince you, well ....

If the science is settled then it is very bad science at the very least.

Your anonymous poster's analysis of the plausibility that this was a hack is, I think, a little overconfident. This could have been the work of an outsider.

First, not all computer break-ins need to get in and "GTFO" as quick as possible. Some exploited computers can be kept under the control of their exploiters, unbeknownst to their owners, for quite some time. It's not clear how CRU actually discovered they had been broken into, though. Did they find out through their own internal security, or did they simply infer it from the appearance of these emails?

And second, it's not clear that someone who broke into their systems would actually have to do too much work to compile all this info. Isn't it all from emails? They could have collected all the emails pretty readily. It's not as if they'd be stored in random locations throughout the network. As another poster already noted, a lot of the work here could have been done offline.

The argument cuts both ways, if there was so much stuff going on then presumably and whistleblower could have got stuff a whole lot worse.

Sure, but I had in mind people (not necessarily those who were posting on this blog) who say, "This is the result of some anti-scientific crank, who is not above stealing emails. So take all this with a grain of salt."

So my point to that kind of a defense is, "It's pretty bad when some random denier breaks in and grabs whatever he can, and then discovers five really damning things."

Another way of putting all this, anon, is that these things that were released are explosive enough that I could see a whistleblower feeling compelled to release them. I just came across the most damning thing I've yet seen (I'll post in a minute).