The Climatic Research Unit and University of East Anglia emerge from the …

Last week, I wrote an article about Michael Mann being completely exonerated by the inquiry at Penn State, and, at the time, I said that Phil Jones and the Climatic Research Unit (CRU) had been cleared by three inquiries. I got that wrong: at the time it was only two. Yesterday, the third inquiryreported back (pdf). And, yes, the science from the CRU has emerged unscathed. But— and its the sort of "but" that is both big and little—the reputation of both the CRU and the University of East Anglia (UEA) have taken a battering as the ugliness of character was there for all to see.

The latest inquiry was led by Sir Muir Russell, who had complete freedom to both set the terms of reference and recruit people to the inquiry. The members of the inquiry consisted of five people from both public and private sectors, all distinguished in their fields. The inquiry proceeded mostly by following document trails. In the initial stages, the inquiry determined that it would be as open as possible, but, in order to not cover the ground of the previous inquiries, would focus on the behavior of CRU and UAE staff. Here's how the inquiry was laid out.

Given the nature of our remit, our concern is not with science, whether data has been validated or whether the hypotheses have survived testing, but with behaviour; whether attempts have been made to misrepresent, or "cherry pick" data with the intention of supporting a particular hypothesis, or to withhold data so that it cannot be independently validated, or to suppress other hypotheses to prevent them being put to the test.

In particular, the inquiry focused on whether researchers manipulated data or avoided Freedom of Information Act (FoIA) requests. To help focus the inquiry, they asked for—and received—written submissions, which, depending on the legality and confidentiality of the submission, were made available on the website. These submissions were then boiled down to a series of questions.

One thing that became clear is that the CRU has been a victim of changing circumstances. When the CRU began, climate science was a backwater as far as the public was concerned, and the CRU beavered away without anyone noticing. However, growing concerns, due to the findings of various researchers, pushed climate science onto center stage. When it turned out that the CRU was in the forefront of supplying data for the more public figures, it became a focus of attention for the denialist community.

In the meantime, digitization had hit, and suddenly it was possible to collect far more data than previously, and, not surprisingly, the folks at the CRU (and elsewhere—this is a problem that has hit many branches of science) didn't really know how to handle it. So, as expected, they handled it badly. My impression is that data archiving practices have changed substantially in just the last year, so the CRU, along with other organizations, were aware that this was becoming a problem and took steps, but it really was too late by then. Add to that the fact that the Freedom of Information Act caught everyone unawares, suddenly making many a scientist's records subject to potential scrutiny.

The third strike is something that I really hate to admit. The CRU simply didn't want to engage with either the public or their "critics." In some respects, I understand why the scientists would get sick and tired of dealing with the incompetent bumblings of the likes of Christopher Monckton. But Phil Jones in particular didn't like blogs, or responding to blogs. And frankly, I am appalled by that attitude. I spend a good portion of my free time attempting to communicate science, and here we have a major contributor—who also happens to be pretty damn clever and in a great position to put forth his work—refusing to participate. The world has moved on, and if your science is front and center, you have take the microphone and sing.

Instrumental data

So with that background, let's move on to specific allegations:

That CRU prevented access to raw data.

That CRU adjusted the data without scientific justification or adequate explanation. Some allegations imply that this was done to fabricate evidence for recent warming.

That CRU inappropriately withheld data and computer code, thus inhibiting others from checking the conclusions of the CRUTEM analysis.

Most of the submissions either implied this or made explicit statements. Most specific were the claims of McKitrick, claims not covering the availability of the raw data, but rather the lack of availability of the list of additional temperature stations used by the CRU. On the other side, there were several submissions supporting the scientific practices of the CRU, and noting that the raw data was publicly available.

I loved the way the inquiry dealt with this: "In order to test the principal allegations of withholding data and making inappropriate adjustments, the Review undertook its own trial analysis of land station temperature data. The goal was to determine whether it is possible for an independent researcher to (a) obtain primary data and (b) to analyse it in order to produce independent temperature trend results. This study was intended only to test the feasibility of conducting such a process, and not to generate scientific conclusions." In other words, if we can do it, anyone can.

They found that the data was readily available at at least three different websites. They downloaded the data, selected every station that had an adequate amount of data and performed some smoothing and spatial averaging operations on them. In effect, they replicated the CRU's main research results, producing nearly identical instrumental temperature records, in very little time. This is something that seemed to be beyond "expert critics" such as McIntyre and McKitrick. To quote the inquiry, "The test is therefore sufficient to demonstrate that, with respect to the declared method, the CRUTEM analysis does not contain either error or adjustments which are responsible for the shape of the resultant temperature trend. A researcher can evidently produce a study which would test the CRUTEM analysis quite precisely, without requiring any information from CRU to do so."

So, in the very broadest terms, the CRU did not, and, in fact, could not prevent other researchers from accessing the instrumental data record. Further, the CRU has not manipulated the data to obtain a preordained result. And, at the most basic level, the code is simply not required. However, this is not replicating in every detail the CRU's work, so the question of code and data sharing was examined in more detail.

The question of station identifiers was key to this, and is problematic on several levels. First, the provision of station identifiers under an FoIA request was subject to unwarranted bureaucratic delays from both the UEA and the CRU. It was eventually handed over, however. The more difficult task is taking the station identifiers and matching them to actual stations used for different temperature reconstructions: there is no coherent international naming scheme. In spite of this, using CRU records and records from one other agency was sufficient to identify 90 percent of stations uniquely. The CRU said that the remaining 10 percent can be identified using Japanese records.

The point is that the amount of work required to do this makes the whole process rather error-prone. As a result, the inquiry found that, although the information was publicly available, the CRU should have been helpful and supplied a list that uniquely identifies the different stations. In addition, they recommend the following:

It would benefit the global climate research community if a standardised way of defining station metadata and station data could be agreed, preferably through a standards body, or perhaps the WMO. As example an XML-based format which would make the interpretation, use, comparison, and exchange of data much more straightforward.

Without such standardisation there will remain residual problems in issuing unambiguous lists, and assembling primary data from them. We feel it would be in the public interest if CRU and other such groups developed a process to capture and publish a snapshot of the data used for each important publication.

On the question of code sharing, the inquiry was entirely unsympathetic. It took them two days to reproduce the data extraction and statistical routines to reproduce the CRU's work. The rest is the donkey work of checking each station's record for discontinuities and malfunctions.

Finally, the inquiry reviewed claims that the CRU had published results based on fraudulent data from China. Here, I would say that the CRU got lucky. It turns out that the data was not what it purported to be, but, when that came to light, the CRU reinvestigated their findings using a different data set and found that it didn't change the conclusions of their original paper. Once again, the CRU proved less than forthcoming. It is clear that they initiated their new investigation right after the claims came to light, but they refused to say anything of substance about it until they published the results of their new investigation.

Chris Lee
Chris writes for Ars Technica's science section. A physicist by day and science writer by night, he specializes in quantum physics and optics. He Lives and works in Eindhoven, the Netherlands. Emailchris.lee@arstechnica.com//Twitter@exMamaku