Co-Opting the US Department of Energy

Maxim Lott of Fox News has an interesting article drawing attention to the co-opting of US Department of Energy funders by CRU and their associates – co-opting in the sense that the US Department of Energy totally failed to ensure that their grant procedures complied with US federal policy for requiring grantees to archive data.

In the article, Lott quoted extensively from Jones’ emails between 2007 and 2009 in which he assured correspondents that the US Department of Energy had assured him that he didn’t have to archive data:

Making that case in 2009, the then-head of the Research Unit, Dr. Phil Jones, told colleagues repeatedly that the U.S. Department of Energy was funding his data collection — and that officials there agreed that he should not have to release the data.

“Work on the land station data has been funded by the U.S. Dept of Energy, and I have their agreement that the data needn’t be passed on. I got this [agreement] in 2007,” Jones wrote in a May 13, 2009, email to British officials, before listing reasons he did not want them to release data.

Two months later, Jones reiterated that sentiment to colleagues, saying that the data “has to be well hidden. I’ve discussed this with the main funder (U.S. Dept of Energy) in the past and they are happy about not releasing the original station data.”

A third email from Jones written in 2007 echoes the idea: “They are happy with me not passing on the station data,” he wrote.

I was contacted by Lott and drew his attention to correspondence in 2005 between Warwick Hughes and the US Department of Energy, which was reported at CA in October 2005 here. Earlier in February 2005, Jones had famously refused Warwick Hughes as follows:

Even if WMO agrees, I will still not pass on the data. We have 25 or so years invested in the work. Why should I make the data available to you, when your aim is to try and find something wrong with it. There is IPR to consider

In my October 2005 post, I reported Hughes’ email to Tom Boden of the US Department seeking Jones’ data, together with Boden’s response advising Hughes that the terms of Jones’ grant did not require him to provide the funding agency with the data. Curiously, in the early 1990s, under earlier Department of Energy contracts, Jones had provided the US Department of Energy with a complete station data base which was published online by the DOE. (The University of East Anglia has never provided any theory of how alleged confidentiality agreements from the 1980s remained “actionable” given that Jones had either not been prevented by these agreements from providing station data to the US Department of Energy for their online publication or had done so without complaint by the NMSs at the time.)

The October 2005 post is well worth re-reading as it reviews US federal policy which would appear to have required the DOE to ensure that grantee CRU archived data. It also quotes policy from UNFCCC which also clearly required NMSs to have ensured that a system for archiving station data was established. The position of Nature and other defenders of CRU as justified due to elusive confidentiality agreements entails that IPCC participants who rely on these supposed agreements have failed to live up to these UNFCCC obligations. In my opinion, Jones, in his capacity as IPCC AR4 Coordinating Lead Author, ought to have reported such NMS consent refusals to IPCC participants as breaches of UNFCCC commitments. Jones’ failure to do so is hard to separate from his expressed desire as CRUtem proprietor to prevent critics (for reasons that remain unclear) from examining this data.

In my October 2005 post, I wondered at the tactical wisdom (from the Team’s standpoint) at obstructing access to this (and other) data, noting then (as on numerous subsequent occasions) that I did not necessarily anticipate large changes in the temperature history, but equally, given the wide public interest, why the record should not be made available for scrutiny.

The whole point of the thread (and many like this) is that the data on which conclusions are based are not available, and in defiance of the stated policies of several parties involved in funding the research and disseminating the results.

Replication must begin with access to the data used otherwise it isn’t science.

And a lot of analysis has been done here – the briefest exploration confirms that.

Why do you think it is your place to decide the remit of climateaudit, steve mcintyre, or any other interested party? There are a class of people who find the data interesting. They find the data interesting for a variety of reasons:

1. to confirm what they believe
2. to find errors
3. to satisfy their curiousity

There are other classes of people who find the whole process of government sponsered science
interesting. These people may only be interested in seeing that the government processes we
fund are actually followed. That is a legitimate interest. You dont answer the questions about how processes were or were not followed by attacking the watchdog or by suggesting that the
watchdog should do something else.

Personally, I am interesting is making sure that the science my tax dollars funds, follows
a well defined process. The same way I am interested that the money spent on defense weapons
is properly spent. Nobody ever suggested to ralph nader that he should build his own damn
cars rather than criticize motor city. When defense teams look at old cases to free innocent men convicted of crimes, nobody suggests that they should become prosecutors. No, we welcome investigations into processes so that we can build better systems. At least some of us do.

Has this website now sunk to hitting the people behind data you do not like rather than researching the data that proves them to be incorrect?

Or is this impossible because you have done no research?

How can one dislike data that are unavailable? The issue, were you to actually read the article, is that the data were not available and Phil Jones to refused to make them so. Without that, as others have pointed out, the conclusions advanced by Jones and his associates can’t be replicated. Their feelings might be hurt that other scientists may be dubious of their apocalyptic conclusions, or that experienced statisticians doubt their skill at obtaining empirically meaningful results, but, in fact in all the sciences – except climatology – the ultimate balm to bruised egos is to be able to say “I told you so.” That cannot happen unless the data, physical assumptions, adjustment methods, and analytical algorithms and the logic behind them all are not available for analysis.

The sole apparent benefits of not publishing the data and the rest of the list above is that 1) the researcher is not embarrassed by another analyst pointing out a drop sign or unphysical result in a badly framed calculation, 2) the researcher continues to draw grant money, and 3) the bureaucrats of the grant-dispensing agency are not embarrassed.

But their own actions shall you known them , in other words the ‘quality’ of the data can be strongly implied by the actions of the person creating it . Especial when the the data its and origins is so hard to even get your hands on given the ‘Teams’ approach keeping things within their group or even ‘losing’ it

Steve,
Thanks for your persistent efforts to highlight bad behaviour among IPCC and CRU leadership.
I appreciated your link to this long-brewing impropriety.

Is it correct to say that the DOE-funded ‘HadCRU recipe” is still a secret?
It seems that they have made a ‘pro-forma’ release of some data, but it is still insufficient to deduce and test their ‘secret’ recipe?

The disability-to-replicate (caused by their non-disclosure) seems to me to be the core issue, no?
RR

In the original post in October 2005, I quoted various policies that mandated DOE to require Jones to archive data e.g. the US GLobal Change Research Program directive to fnding agencies on grant language:

SUGGESTED DATA PRODUCT REQUIREMENT FOR GRANTS, COOPERATIVE AGREEMENTS, AND CONTRACTS
Describe the plan to make available the data products produced, whether from observations or analyses, which contribute significantly to the grant results. The data products will be made available to the grant without restriction and be accompanied by comprehensive metadata documentation adequate for specialists and non-specialists alike to be able to not only understand both how and where the data products were obtained but adequate for them to be used with confidence for generations. The data products and their metadata will be provided in a standard exchange format no later than the grant final report or the publication of the data product’s associated results, whichever comes first.

and the following statement from DOE:

OSTI {office of Science and Technology of DOE] assures access to the results of DOE research by the scientific research community, academia, U.S. industry, and the public.

In 2005, attitudes towards providing data had hardened. I made the following observations in October 2005, which seem sensible to this day:

As to why the Hockey Team is so keen on data secrecy, I have no idea. It makes them look bad for very little point – if I were coach of the Hockey Team, I’d tell them to stop acting like prima donnas, archive their data and get on with things. But that’s just me.

If I were an American citizen, I’d want to know why the bureaucrats who have been funding all this work have failed to include adequate contractual language in Jones’ contracts to ensure that there was archiving, just like there is for so many other D.O.E. programs.

If I were advising a congressional oversight committee, I’d sure add this to my list of questions.

If I were a D.O.E. manager who had funded these primadonnas, I’d tell them that, regardless of the technicalities of past contract language, if they wanted another nickel from me, they’d better get their asses in gear and set up a proper data archive – and they’d better do it by yesterday so I didn’t look like an idiot if I heard from some congressional committee.

Needless to say, up to Climategate and FOI appeals in the UK, little had changed and, indeed, in some cases, matters had got worse.

I remind readers once again that I’ve consistently discouraged readers from thinking that Jones’ obstruction implies that there is some smoking gun in the CRUtem data. Quite frankly, I’ve always been a bit puzzled as to why Jones was so adamant about archiving data. My own theory has been that it was to avoid showing precisely how little work went into it. For example, here is my contemporary (2009) reply to Peter Webster’s comment mentioning that Jones had sent him the email:

Having said that, Jones’ obstruction is a disgrace as is the acquiescence in his obstruction by the Community. Every day that this goes on, Jones embarrasses the “Community” and does a disservice to the cause that he’s advocating.

As I’ve said before, readers should not expect that there’s some smoking gun in the secret data set. I don’t expect one. My own opinion, as I’ve said before, is that Jones doesn’t want to reveal the negligible amount of quality control that they do.

Gavin Schmidt says that GISS spends 0.25 man-years on their index. CRU probably does the same. They probably have old, old programs that like GISS, sort of work, but they’d be embarrassed to show anyone. And for running these ancient programs once a month on data prepared by others, they get lots of money.

Once their data and methodology is disclosed, my guess is that it wouldn’t take me very long to put a CRU emulator online and call into question why they get $1 million per year or whatever to calculate their index.

I think that the obstruction is monetary and has nothing to do with climate change. (I could be wrong.) HAving said that, the Community should not tolerate Jones’ behavior any longer.

I made similar comments on other occasions.

Even in the October 2005 article, I did not encourage readers to expect that close examination of CRUTem would lead to a “material change” in the index. My own interest was in the proxy reconstructions. Temperature indices were topics that interested others. However, I was sympathetic to their efforts to access data and quite prepared to use whatever influence I had to support those efforts as expressed in 2005 as follows:

I am extremely sympathetic to Warwick Hughes’ attempt to get access to the station data used in the CRU temperature reconstructions. I cannot say whether a close examination of that data set will lead to any material change in the view of temperature changes over the past century, but checking the dataset seems a worthwhile exercise. Jones’ secrecy seems unacceptable to me. In particular, the bureaucrats who have written contracts that permit Jones to withhold the data should be brought to account.

RuhRoh asks – “Is it correct to say that the DOE-funded ‘HadCRU recipe” is still a secret?”
I have seen no evidence that the DoE is funding the Hadley Centre/UK Met Office. DoE funding that I am aware of is to Prof Jones – and his CRUT2 series has not been updated post 2005. From 2006 the HadCRUT3 series was born with Prof Jones a minor author – last in the list – so I assume he still has some role in data for HadCRUT3.
The UKMO has never published anything remotely resembling the DoE books documenting the Jones et al methodology that were published up to 1991.
I always thought the UKMO took the management of CRUT away from Prof Jones because key people were unhappy with stewardship.
Who knows where all this will go and what we will ever learn. I have just posted;
“US Dept of Energy – should come under more scrutiny”

in the climategate 1 files there is a mail where they discuss getting a grad student for crutem3

also, it would be great if you put together all your correspondence in a chronological order.

I will note this. Often we see the defense made that skeptics really never cared about the data. we see the charge that they never published anything. and we see the charge that they wouldnt pay for data. I think your story and Geoff’s story is a very nice prequel to the entire history.

Just wondering if Mr. Mosher read this thread RE: his FOI/EIR requests? It’s an interesting discussion about who determines what is considered a confidentiality agreement. Phil lists just about everything but a secret handshake. Altho that doesn’t mean there isn’t one . . .

As Phil says, “The agreements that we’re talking about are not really confidentiality agreements that you’re talking about.”

Hi Matt,
My comments were tongue in cheek! The agreements that we’re talking about are not really confidentiality agreements that you’re talking about. Lots are unwritten agreements that we make scientist to scientist. Where there are written agreements they are signed between me (or previous Director’s
of CRU) with other academic institutions, which were not with their central administration (but again a sub part). CRU doesn’t initiate these, but if the other side wants it and it will help us do some work then we go ahead and sign. There is never any obligation on CRU or UEA. They are generally
about agreeing to work together on something.
The agreements Dave is talking about are ones that relate to us not making climate data available to third parties, which we have got from a National Met Service.
FOI is causing us a lot of problems in CRU and even more for Dave, as he has to respond to them all.
It would be good if UEA went along with any other Universities who might be lobbying to remove academic research activities from FOI. FOI is having an
impact on my research productivity. I also write references for people leaving CRU, students and others. If I have to write a poor one, I make sure I get the truth to the recipient in a phone call. I’m also much less helpful responding to members of the public who email CRU regularly than I was 2-3 years ago.
I’ve seen some of what I considered private and frank emails appear on websites. Issue here is blogsites have allowed these climate change deniers to find one another around the world.
Cheers
Phil

This does present something of a ‘issue’ in terms of drafting a response and dealing with any potential follow up request/query regarding our practices in this regard. I
wonder if whether said policy was in force at the time the agreements were entered into would be a way around this… the request is for current policies clearly…. I will
enquire further with Matt Hume….

From: Palmer Dave Mr (LIB)
Sent: Wednesday, August 05, 2009 3:23 PM
To: Hume Matthew Mr (ACAD)
Cc: Walker Alan Dr (ACAD)
Subject: RE: FOI/EIR request [FOI_09-69]
Matt,
Thanks very much for this… You have given me a bit of a conundrum on how to respond but we do at least have something to work with….
What policy are you actually quoting from and is it publicly available? As the request was for the entire policy, is there any issue with making the policy publicly available? If the policy in regards confidentiality agreements is within a larger document with unrelated material, I am happy to quote but I do think we will need to provide a proper citation…

REDACTEDWe all just had a very good laugh at Phil’s comment “We do sometime ignore the
REDACTEDRegistry advice”… If this is going to have the kind of publicity that you suggest,
REDACTEDI would prefer if you do not quote ANY of his answer to question 1.

REDACTEDThe UEA actually has a very strict policy on entering into confidentiality
REDACTEDagreements, however as Phil so blithely admits, a handful of academics take it upon
REDACTEDthemselves to foul things up!

REDACTEDAs you will note from points 1 & 2 of our policy; no UEA employee, except members of
REDACTEDour office, has the right to sign anything on behalf of the university – the problem
REDACTEDis that funders/other parties can be sneaky by sending the agreement in the name of
REDACTEDthe academic.

REDACTEDOur policy is:-
REDACTEDSomeone from the Commercialisation & Enterprise Team should approve and sign all
REDACTEDConfidentiality Agreements:
REDACTEDonly our staff have the legal authority to sign agreements on behalf of the
REDACTEDUniversity
REDACTEDall agreements should be between the University of East Anglia and the party
REDACTEDrequesting the agreement (not an individual academic or school)
REDACTEDwe will negotiate with the other party on any issues within the document that may be
REDACTEDcontentious
REDACTEDby doing this we will ensure you the best protection of your IP rights
REDACTED(In special circumstances, authorisation may be obtained from the Commercialisation
REDACTED& Enterprise Team allowing you to sign the agreement yourself. Such authorisation
REDACTEDmust always be obtained in advance, will only be valid for a specific instance, and
REDACTEDthe standard university agreement must be used without amendment – unless we have
REDACTEDauthorised an amendment)

REDACTEDIn all cases, a copy of the fully signed confidentiality agreement must be retained
REDACTEDin our office.

REDACTEDFYI – we are currently finishing off the final touches to our new intranet pages –
REDACTEDthere will be a page on CDA’s with this info on it.

REDACTEDAlso, I am away on holiday next week (10th -14th), so if you do any more info on our
REDACTEDpolicy regarding agreements etc, please contact Anne Donaldson, one of our
REDACTEDCommercialisation Managers ([1]REDACTED).

REDACTEDA bit of context – in response to a rejection of a request for data, we have
REDACTEDreceived over 50 requests for agreements, data and a combination thereof in
REDACTEDrelation to data sets that CRU maintains/holds. This is pretty high profile
REDACTEDand has been noted in blogs in the Guardian and Telegraph as well as in the
REDACTEDsource of all of this (see: [2]http://www.climateaudit.org). Be assure that
REDACTEDwhatever we state in response to this request is likely to be on the web,
REDACTEDshared and very public within hours of sending….

REDACTEDWe have a request from another individual exactly the same as below so there
REDACTEDwill be multiple recipients of the answer we give.

REDACTEDOur deadline for a response is 21 August but as I’m on hols commencing 17
REDACTEDAugust, the ‘effective’ deadline is 14 August.

REDACTEDThe first part of the query will be answered in line with the answer given to
REDACTEDother requesters for the agreements.

REDACTEDIn regards the second part, I will need some assistance as noted below

REDACTED. A copy of policies and procedures regarding employee responsibilities
REDACTEDregarding entering into confidentiality agreements. REDACTEDns = “urn:schemas-microsoft-com:office:office” />
REDACTEDSteve, do we have any contracting policy on this? Phil – anything with CRU on
REDACTEDresponsibilities regarding entering agreements on behalf of CRU? I don’t think
REDACTEDwe wish to state that we don’t have any policies or procedures in place, but
REDACTEDI’m not sure what to actually put here…
REDACTEDI don’t think there is anything – if there is I’ve never seen it. People in
REDACTEDCRU (not just me) enter into
REDACTEDagreements about data and/or writing papers and getting involved in projects.
REDACTEDUEA signs research
REDACTEDcontracts for us. UEA employees do the work, but UEA administers the grant.
REDACTEDThe various
REDACTEDagreements that UEA signs may say things about data access, but it will vary
REDACTEDdepending on
REDACTEDthe funding body. Some are more stringent than others. The Registry goes
REDACTEDthrough these.
REDACTEDThey mostly help the researchers by not letting ourselves sign away any
REDACTEDrights and IPR. We
REDACTEDdo sometime ignore the Registry advice, preferring to fall back on the verbal
REDACTEDagreements we
REDACTEDhave with the funders (their project officer). If we ever have a problem, we
REDACTEDprobably wouldn’t
REDACTEDwork with them again. This has happened with some scientists I have
REDACTEDcollaborated with in the past.

REDACTED. A copy of policies and procedures regarding employee responsibilities
REDACTEDregarding the preservation of written agreements.
REDACTEDAh, records management rears its head.We have a general statement on our
REDACTEDwebsite regarding our responsibilities for RM but we do lack any overarching
REDACTEDrecords retention schedule or policy Phil, does CRU have anything in-house?
REDACTEDCRU has nothing in this regard.

REDACTED. A copy of policies and procedures regarding employees entering into verbal
REDACTEDagreements.
REDACTEDSee question 1; same issue here although more likely to have a nil response
REDACTEDhere consequences of that?
REDACTED As said in the 2 pager we’re working on, we put some of the agreements in
REDACTEDthe letters we
REDACTEDwrote to Met Services requesting data (some of which we paid for).
REDACTEDThere has been a lot of time and effort gone into making these contacts. It
REDACTEDseems as
REDACTEDthough this counts for nothing.
REDACTEDAgain – unlikely to be anything. People agree things with other academics at
REDACTEDmeetings.
REDACTEDThis is how science works.

REDACTED. A copy of instructions to staff regarding compliance with FOI requests.
REDACTEDWe have web guidance that can be referred to, and a brochure that I distribute
REDACTEDthat could go here. and a statement regarding the training on offer
REDACTEDI’m not sure you want to go down this route!

REDACTEDPursuant to the Environmental Information Regulations, I hereby request
REDACTEDthe following information in respect to any confidentiality agreements
REDACTEDaffecting CRUTEM station data involving station data in NIGERIA,
REDACTEDNETHERLANDS, NORWAY, NEPAL,NAURU
REDACTED. the date of such agreement;
REDACTED. the parties to the agreement;
REDACTED. a copy of that part of the agreement that prevents further
REDACTEDtransmission
REDACTEDof the data to non-academics or others
REDACTED. a copy of the entire
REDACTEDagreement

I don’t know if this branch of “science” follows the practice set out in the publication of medical papers, if it does, the inclusion of Prof Jones in last place on a list of a paper’s authors, simply reflects him as head of department and may have had no input into the paper at all.

Standard practice in my field (molecular biology) is first author did the work and last author was the head of the group – who usually got the grant and was responsible for editing final versions, submission etc. I only ever looked at first and last authors to get an idea of who was involved and to see Prof. Jones as last author signifies a lot more than minor involvement to me.

There is another aspect to the whole DOE funding thing. The money to CRU actually was a big slush fund for them to do whatever they wanted, and the official task(s) listed on the grant(s) actually took very little of their time. While there is expected to be some play in the work done under a grant, I have known people who went off and did something completely different on a long term NSF grant in USA and got their grant terminated, and when I got DOE grants for 7 yrs at Argonne Lab I was expected to AT LEAST deliver what I promised (extra was extra). Very fishy. It is also odd for DOE to be funding a long term project like this in another country. Odd.

Craig. as I said on the other thread, the quote from Jones email 1577:

Any work we have done in the past is done on the back of the research grants we get – and has to be well hidden. I’ve discussed this with the main funder (US Dept of Energy) in the past and they are happy about not releasing the original station data.

Appears to point to the DoE knowing that CRU was using the ongoing grants as a general fund for whatever they wanted to do. What Phil appears to be saying needs to be hidden is that the research is done “on the backs” [diverted from ~ me] the grants. The cited withholding of the station data seems just to be an agreed upon strategy with the approval of the DoE to maintain general opacity lest a trail appear to the actual use of funds. This is a small stretch from the language above, but not much of a stretch. We’ll see how forthcoming the DoE will be with the current FOI requests.

One explanation would be that the funders understood that the true output of CRU and related institutions was nothing as boring as averages of anomalies of the modern instrumental record but the picture that alone seemed to prove that recent temperatures were unprecedented: the hockey stick. They weren’t interested in the detail as long as the Team promoted and defended the ‘dangerous blade’ – and this was made clear to the recipients.

All the way back to (at least) Reagan. That’s five presidential administrations. One would think think in all that time, somebody like an auditor, an IG, or a political operative looking for $600 toilet seats would say ‘we keep giving these guys money but we have nothing to show for it’.

The Carbon Dioxide Information Analysis Center (CDIAC) of the U.S. Department of Energy has been heavily involved in global warming research since the early 1980s and began their sponsorship of Phil Jones and CRU at that time. Station data and documentation for the early studies(TR022 and TR027) on 2873 stations (NDP-020) was made available around 1990. Since then, there have been several iterations of the dataset, including an increase in the number of stations from 2873 to 5129. However, to my knowledge, there has been no updating of the archive on station documentation. Limited updating of the archive of station data was done in 1994 but again, to my knowledge, nothing in the last decade – thus almost nothing since IPCC came on the scene and transformed the politics of the issue.

This leaves it open that the DOE got something for their money before 1994. On the original interest in CO2 and the carbon cycle, from around 1978, see Freeman Dyson’s memories of Oak Ridge that I brought to CA’s attention three months ago. The work in Carter and Reagan’s day was surely on the level. As Steve implies, only as the IPCC came on the scene did things go wrong.

I deal with people in various personal predicaments all the time and for me it seems clear as daylight that either they don’t have the data (or some substantial portion of it)or the data cannot be retrieved, which amounts to much the same thing. Of course, this remains my own personal opinion and is not proof.

Knowing that an admission to this effect would be absolutely devastating for their reputations, not to mention ‘the cause’, they have turned to obfuscation and denial in the somewhat misguided hope that it will simply ‘go away’.

But the internet has changed things. NOTHING remains hidden for very long. One will inevitably be found out!

@ Kohl
Jones admits in the emails that he doesn’t have the data?
Search the Foiaa emails jones Britton and you will see?
Steve
You have always given the Fiddlestick Team the benefit of the doubt and you were right to do so. However this latest release shows something both insidious and corrosive going on. If Honest Phil can say, I paraphrase,
Don’t tell him we haven’t got the data tell him it’s subject to confidentiality agreements. If he try’s FOI we’ll say it’s vexatious.
The above is an extract of just one of thousands of emails where the tone and tenor is quite frankly disgusting.
I wish Honest Phil and The Fiddlestick Team no harm, just time to come clean because it ain’t cricket.

Let’s say researchers have found a cure for a “OMG!!! WE ARE ALL GOING TO DIE1!!” kind of problem, like heart-disease or cancer. Let’s say statins or “antineoplastons”. Then any “rivaling” researcher groups – or any person from the public – will want to look into the research data, and they have every right to do so. The biggest problem in medical sciences is surely plain old confirmation bias, but there are larger incentives to fudge odds-ratios, make the effect size appear bigger and downplay side-effects. In medical science this is a recognized problem and data has to be publicly available – especially because medical problems are a matter of life or death. Why should climate science get more slack, when they claim that climate is matter of life or death?

Explanation is seldom given and certainly no promise of explanation is given. But the quality of the final result, especially for the interested reader with limited time, stands as testimony that Steve’s policy works well, if not wonders.

Note this comment itself is eminently snippable, especially if the host does wish to explain. Maturity for me is to write one’s best and not complain if it’s considered surplus to requirements, governed by the greater good.

It’s not necessarily personal: I recently had two attempts at posting a short comment that merely corrected a previous comment of mine disappear into thin air. After I waited a while, my third attempt went through, at

Even then I managed to spell Donna Framboise’s name wrong! She is the owner of http://noconsensus.org, not to be confused with http://noconsensus.wordpress.com. The latter is the Air Vent, owned by Jeff Id, whose site, along with Tallbloke and CA, is the subject of the US DoJ’s investigation that apparently prompted the Tallbloke raid.

I think there are two reasons why the CRU has not released the data.
1. My first point is similar to Steve’s point which is that they were embarrassed by how little work went into it. I would go further and suggest it was to hide a general lack of professionalism. It is quite easy to calculated global temperatures from the currently available data bases; what is difficult is to maintain an audit trail which preserves the original data, intermediate data sets using slightly different algorithms, identification of data sets rejected and the reason for it, etc. The indications are that the CRU did not do these things.
2. Many countries, including the UK, charge for meteorological data (this I know from firsthand experience in many countries). That explains why some countries insisted on confidentiality agreements; the fact that the CRU could not find these agreements reinforces my first point.
I should add that whilst I believe it is a disgrace those whose work is used to justify the expenditure of trillions on combating climate change have been so cavalier with their data, that the costs of going back to national meteorological services for the original data would be justified, I don’t think the outcome in terms of global temperature would be much different.

Ron, As I’ve posted a couple of times, Australia at least did not seem to have a limiting agreement with CRU or UEA at many relevant times. Rather, the tendency seems to be for a timid Australian Government to fail to investigate if some at least of the Australian data were altered and distributed by CRU contrary to express clauses in the Australian Copyright statement as it stood from time to time.

2. Many countries, including the UK, charge for meteorological data (this I know from firsthand experience in many countries). That explains why some countries insisted on confidentiality agreements; the fact that the CRU could not find these agreements reinforces my first point.

This is true. I know from experience that GISS charges for detailed terrain maps, for example (you can get the coarse maps for free, however). GISS’ terrain data is not expensive (I don’t recall the actual cost since the coarse data was sufficient for my needs at the time), and readily obtained from their website.

What should have been done, however, was supply all the data not covered by such agreements, then point requesters to each of the sources for data covered by any agreements. As you note, the fact that said agreements couldn’t even be found indicates their lack of professionalism as well.

The position of Nature and other defenders of CRU as justified due to elusive confidentiality agreements entails that IPCC participants who rely on these supposed agreements have failed to live up to these UNFCCC obligations.

Quite interesting to contrast Nature’s position on CRU data with their recently reported statements on the bird flu data (see here for the BBC news report).

Apparently the US gov are demanding certain details of a recent study into the ability of bird flu to spread through ferrets to humans be redacted in case it could be used by terrorists; Nature and Science are taking a stand, insisting that researchers MUST have access to the redacted data.

I haven’t looked at the bird flu aspect in any depth and have no strong view on it, but it is astonishing to see the difference between this approach and their handling of the CRU data access, where they were all too quick to defend refusal of access to data by researchers.

I wonder if they are learning from their past errors, or if they are just showing staggering and brazen hypocrisy?

Unless someone can show otherwise, I dispute the statement “Progress has been made in the SH since AR4, where new tree ring records from the Andes, northern and southern Patagonia, Tierra del Fuego, New Zealand and Tasmania (Boninsegna et al., 2009; Cook et al., 2006; Villalba et al., 2009)” … in respect of Tasmania at least.

As observed by the late John Daly, the main trees studied in Tasmania grow in the wet, cold, mountainous west, where there are few (really nil) reliable long term temperature records. The calibration was done with stations some 150 km or more east, on the hotter, drier lowlands.http://www.john-daly.com/hockey/hockey.htm see his Exhibit 8, though the whole article is relevant.

Unfortunately, though we can core more trees and erect more weather stations, we cannot calibrate to past temperature in a way that avoids serious questions without answers. Ed Cook’s studies on Huon Pine predate AR4, and I can find no relevant Villalba paper later than AR4.