CRUTEM3 data release (except Poland)

The entire CRUTEM3 database of station temperature measurements has just been released. This comes after a multi-year process to get permissions from individual National Weather Services to allow the passing on of data to third parties and from a ruling from the UK ICO. All the NWSs have now either agreed or not responded (except for Poland which specifically refused). Since the Polish data is a such a small fraction of the globe (and there are a few Polish stations in any case via RBSC or GCOS), this doesn’t make much difference to hemispheric means or regional climate. These permissions were obtained with help from the UK Met Office (who have also placed the station data on their website in a slightly different format) and whose FAQ is quite informative.

Oh good. I guess we can expect a zillion temp reconstructions from those so desparate to get their hands on it. Can’t we? I can just see the thousands of thanks sent to Phil Jones and CRU for all the hard work….

Well done! I’m pretty sure that there will still be those that claim the data is fudged. But in reality, for the sceptic side, IMO, this fight was over a couple of years ago when some prominent sceptics actually did do their own analyses and came up with basically the same answers.

But still, this is a very big step in hopefully calming things down a bit, so that we the people can actually get going on making some decisions.

Although Jones agrees that the data should be made publicly available, he says that “it needs to be done in a systematic way”. He is now working to make the data publicly available online and will post a statement on the CRU website tomorrow to that effect, with any existing confidentiality agreements. “We’re trying to make them all available. We’re consulting with all the meteorological services – about 150 members of WMO – and will ask them if they are happy to release the data”, says Jones. But getting the all-clear from other nations could take several months and there may be objections. “Some countries don’t even have their own data available as they haven’t digitized it. We have done a lot of that ourselves”, he says.

August 2009.

Once the data become publicly available, Jones wants McIntyre to produce a global temperature record. “Science advances that way. He might then realize how robust the global temperature record is”, says Jones. Asked if he would take on the challenge, McIntyre said that it’s not a priority for him, but added “if someone wanted to hire me, I’d do it”.

From what i understand, they didnt release it because they wanted to, but because it was imposed to them no June 23rd by the Information Commissioner after a FOI request.

“This comes after a multi-year process to get permissions from individual National Weather Services to allow the passing on of data to third parties and from a ruling from the UK ICO”

The ruling from UK ICO clearly states that permission by individual countries doesn’t matter, data must be made public.

And, on that ground, i don’t understand why Poland data wasn’t released… the ruling clearly states that the confidentiality agreements don’t hold.

“As has been noted in relation to the consideration of the other exceptions, there is very limited evidence available to the Commissioner from prior to, or subsequent to, the Met Office/UEA consultation exercise of the reasons why any of the NMSs might not consent to the disclosure of the information that they supplied to UEA. He is consequently not satisfied that it is more probable than not that disclosure would adversely affect the interests of the information providers and that regulation 12(5)(f) is engaged.”

“The Commissioner requires the public authority to take the following steps to ensure compliance with the Act:
disclose all of the information contained in dataset A and dataset B to the complainant.”

If Poland data is included in dataset and/or dataset B, it must be made public, even if it has negligible influence on global temperature, it’s a matter of principle.

[Response: There are two separate issues here. The FOI request and ICO ruling was for a specific subset which was for near-tropical data. So that wouldn’t involve Poland. The full data set linked to here is not in direct response to the FOI requests. Any subsequent FOI for the Polish data would have to be assessed again – and if the Polish NWS has made it clear that release would have direct consequences, the ICO might make a different conclusion (since his ruling was not against confidential data in principle, but on the lack of specified consequences). – gavin]

Would it be wildly off the mark to think the Polish refusal has anything to do with protecting the denialism of Vaclav Klaus, President of the Czech Republic and current President of the European Union?

Congratulations to CRU on the great effort to release the data. The CRUTEM3 data release should solve all of those problems with people complaining about that data being withheld. Just like all of the evidence should prove to everybody that RC is right. But I think that the other commenters are correct in saying that the same denialists will still be denialists.

In my efforts to find out why, I am reading “The Authoritarians” by Bob Altemeyer. The free book is at: http://home.cc.umanitoba.ca/~altemey/
Professor Altemeyer says that authoritarian followers have extremely compartmentalized minds. They never check for contradictions and they believe the latest thing their leader said. There is very little hope of changing them, although a university education helps a little.

Benjamin@11 stated:
“The ruling from UK ICO clearly states that permission by individual countries doesn’t matter, data must be made public.”

Would this happen in a world where all endeavours were commercially run and data had a price??
There seems to be a lot of politically confused people around these days, they can’t decide whether they are free marketeers or card carrying commies.

Benjamin@11 stated:
“If Poland data is included in dataset and/or dataset B, it must be made public, even if it has negligible influence on global temperature, it’s a matter of principle.”

So the whole business wasn’t about science, it was about being a pain in the back side. Which has long term consequences.

I’m from Poland. Polish decision has nothing to do with Klaus, or with stubborn denialism of both Polish major political parties which is a fact. The Polish national weather service IMGW has always seen all meteorological data as a business asset that should be tightly protected. Although IMGW is a state-owned institution receiving public funding, Polish public has no free access to current or archive station data. Actually, this is ridiculous because everyone can get the data from weather.com, GHCN or many other sites anyway thanks to WMO agreements. Even images from meteorological radars are published in Poland with 4-hour delay, unless official hydrological alerts are issued – a situation unseen in other countries in the region. But if Poland is presented in the media as the only opposing country – what should happen – I believe it would eventually let the data out.

[Response: This sounds by far the most likely explanation. It was (and remains apparently) the commercialization of scientific information that is the barrier here. – gavin]

It’s only a guess, but perhaps Polish NWO’a refusal to hand over the data might have something to do with this. It it is indeed so, that attitude is counterproductive, if you ask me (and I’m Polish). The emission reduction agreement has to consider the fact that Poland, having vast coal reserves, relies on coal for energy generation and really cannot afford to switch to other energy sources quickly, but the negotiations won’t be helped by a display of ill will.

dhogaza #15, it is the Information Commissioner of Airstrip One who has unilaterally revoked Trinidad and Tobago Met Office property rights. So it is not clear what rule of law you are referring to. Certainly not international IPR law. Here’s to hoping this has consequences, as any legalized theft should.

Benjamin — “From what i understand, they didnt release it because they wanted to, but because it was imposed to them no June 23rd by the Information Commissioner after a FOI request.”

Phil Jones clearly states in the interview of August, 2009, that he was working on getting the data released, but in a systematic way. If you go to the webpage RC links to, you can see the data released in a systematic way. Where’s the inconsistency? What’s the issue? He followed through on what he said. But if I were you, I’d be organising a whipround to pay McIntyre to do that reconstruction and see if he follows through on his word.

It must be soul-destroying dealing with all that bureaucracy and inconsistent nonsense. I really feel for Phil Jones, getting all the blame, all the work, and being the scapegoat for every bureaucrat covering their behind.

The ICO ruling is positively Byzantine. It actually bases its ruling on information that was not available at the time of the original decision! If this leads to NMSs withholding information in the future, then all the whole silly exercise will have achieved is the loss of valuable data.

The ICO should certainly have concluded that the original decision was correct, even if circumstances have changed now. But no, they had to be mean. How petty of them!

I don’t think there will be fallout from Trinidad and Tobago. But what about all those other NMSs that simply didn’t respond?

The ICO may be technically right in that it is enough to gain the consent of a representative sample of NMSs, but in my opinion the ICO failed to properly account for the potential withholding of future data.

Didactylos, I can see in my mind’s eye your eyes rolling… yes this is Hitchhiker’s Guide bad. The real badness IMO is not the consequences for future data availability — that’s way too instrumentally thought. No, it’s what this does to respect for the law internationally, which is already at a low, and now, Britain failing to get its national practices compliant with the treaties they are a signatory to.

“Is this a coincidence?”
Nah. Spencer has become a real crank, but it didn’t just happen today. Do read Stoat (above) and Skeptical Science. Spencer is on the Board of the Marshall Institute; the delightful Heartland bunch (cross bred with Forbes) produces his press release, and our complicit press plays it straight. If it were real science they would probably bring in a nutter for false balance, but since this item is nutty to start with they leave well enough alone. All in a days work for some.

Spencer is on the Board of the Marshall Institute; the delightful Heartland bunch (cross bred with Forbes) produces his press release, and our complicit press plays it straight.

Yahoo news is just a bit of software that sweeps up content from a wide variety of news sources. I don’t think of software being “complicit” in the sense you mean, the algorithm’s blind to questions like “is this media source a lying sack of shit or not?”.

That’s just not the kind of stuff that even advanced AI software can do.

The ICO should certainly have concluded that the original decision was correct, even if circumstances have changed now. But no, they had to be mean. How petty of them!

I don’t think they are being mean and petty. Since the FoI Act was introduced in the UK the ICO has taken a pretty robust stance against bodies which refuse FoI requests without having extremely strong grounds for refusal and it has certainly upset the government on a few occasions. Speaking as someone who has very hawkish views on FoI I think it has done an excellent job, especially given that the Act was worded less strongly than I would have liked with too many exemptions and the deeply ingrained culture of secrecy in many public bodies and government departments.

Maybe in this specific case they made the wrong decision – we will have to see how it pans out and if NMSs do actually withold data in future as a result of this ruling, but looking at it from a wider perspective I would much rather have an ICO which errs on the side of those requesting information than on the side of those who hold it.

On the new NASA data blowing a “gaping hole” in “alarmist” global warming predictions…

…I didn’t think that the major measure of how increasing CO2 concentrations in the upper atmosphere might slow heat energy release to space depended on how CO2 would increase atmospheric humidity and clouds.

Here is the abstract from the Forbes-linked article:

On the Misdiagnosis of Surface Temperature Feedbacks from Variations in Earth’s Radiant Energy Balance

Roy W. Spencer and William D. Braswell

“Abstract: The sensitivity of the climate system to an imposed radiative imbalance remains the largest source of uncertainty in projections of future anthropogenic climate change. Here we present further evidence that this uncertainty from an observational perspective is largely due to the masking of the radiative feedback signal by internal radiative forcing, probably due to natural cloud variations. That these internal radiative forcings exist and likely corrupt feedback diagnosis is demonstrated with lag regression analysis of satellite and coupled climate model data, interpreted with a simple forcing-feedback model. While the satellite-based metrics for the period 2000–2010 depart substantially in the direction of lower climate sensitivity from those similarly computed from coupled climate models, we find that, with traditional methods, it is not possible to accurately quantify this discrepancy in terms of the feedbacks which determine climate sensitivity. It is concluded that atmospheric feedback diagnosis of the climate system remains an unsolved problem, due primarily to the inability to distinguish between radiative forcing and radiative feedback in satellite radiative budget observations.”

[Response: The full paper is here – but there are multiple issues here. First off the Forbes article does not represent the paper well at all – S&B only conclude that short term variations are not useful for constraining sensitivity, not that the models have the wrong sensitivity (though this is was Spencer wants to think). But the analysis in S&B is very poor – there are no error bars shown, they appear to be calculating regressions on smoothed data (without taking into account the decrease in degrees of freedom), they use 100 years of data for the models, while using only 10 years of data for observations (with big differences in the noise level), and the ‘simple model’ used is the same as the one excoriated by Barry Bickmore in a serious of posts. I predict that any re-do of this calculation will not support S&B’s conclusions. – gavin]

@41…Re. Petermann Ice Island. I live in Newfoundland. The island is not floating in the “middle” of the North Atlantic, it is floating down the Labrador Current. So far it is staying east of the Northern Peninsula/Straits of Belle Isle and therefore is probably unlikely to ground there. If it should go through the oilfield areas in anything remotely like its present form, the Hibernia gravity-based structure is likely toast. The FPSO’s in the other fields could likely disengage and move off.

A mitigating factor is that it is melting quite rapidly–there are waterfalls coming down the sides that are pretty spectacular. T’will be an interesting September or so.

@ J Bowers – longform, think birthers/Oily Toads (or whatever her name was).
p.s. well done on landing two good solid ones on McTyres in that Guardian thread. Proabably scuttled away to blog something venomous with added bile about the team now.

As you can see his period is the yearly cycle as opposed to the feedback period that according to him is probably 600 days,but he says specifically:

“Yet, as seen in Figure 2, we are still faced with a rather large discrepancy in the time-lagged
regression coefficients between the radiative signatures displayed by the real climate system in satellite
data versus the climate models. While this discrepancy is nominally in the direction of lower climate
sensitivity of the real climate system, there are a variety of parameters other than feedback affecting
the lag regression statistics which make accurate feedback diagnosis difficult.”

So I have a hunch that he completely ignores the part of the cycle when the oceans are absorbing more than what they are irradiating. Since the irradiation frequency is smaller than the summer winter cycle then any period that is not -λT will ALWAYS create lower climate sensitivity. If he is only using the the summer winter cycle he is basically resetting the feedback in his model every Jan 1.

Basically what he is arguing is that the ocean is emitting more energy than what they are absorbing, (if you ignore radioactive decay) then this is a clear violation of the laws of conservation of energy.

andrew adans: That was my response to this specific incident, but in the wider picture, my opinion is that NMSs should make their data public, and that busy-bodies trying to cause trouble should go and bother those individual services, instead of forcing the CRU and Met Office to waste vast amounts of time and effort in response to a vicious and ideologically driven campaign of harassment through vexatious and pointless FoI requests.

And yes, I stick by my comments about the ICO. They got a fair amount of blame themselves (justified or not) in the various Climategate reports, and my opinion is that it rankled.

As to your own wider view – consider, if you will, someone requesting information about specific people, perhaps even yourself. How would you feel if the ICO managed to twist the regulations to be more “open”, then decided to release your personal information without your consent, because the data holder wasn’t able to show that there were any guaranteed adverse effects heading your way? I don’t believe there really is a slippery slope argument here, but I’m unhappy about the precedent.

Please also bear in mind that without the specifically higher standard applied to environmental data, the original decision would have been upheld without question.