Phil Jones: the Secret Agent in Hawaii

As CA readers know, Phil Jones keeps his CRU data secret. Embarrassingly, the UK Met Office relies on this secret data and says that it is unable to provide this supporting data for the most relied upon temperature data set in the world. Their statements in response to FOI requests as to what they actually hold seem contradictory, but most recently they state that they do not hold original data, but only the “value added version” provided to them by Phil Jones. Whether they are entitled to keep the “value added version” secret is something that their FOI officer is presently considering.

Recently, Anthony Watts discovered that the Honolulu Observatory data, which NOAA and NASA lost track of in the 1980s, continued to the present day.

Anthony observed the substantial difference between trends at Honolulu airport and at more rural sites.

When I’ve done previous benchmarking of GISS data, I’ve usually tried to use relatively isolated stations so that the effect of data inclusions and exclusions could be simplified. Since Hawaii is relatively isolated, it seemed like it would be an interesting exercise to look at the Hawaiian gridcell, to get a preview of whether the “discovery” of a long data set might have an impact at the gridded level.

As so often, when one goes down a climate science rabbit hole, wonderland awaits.

First, here’s a graphic showing an interesting contrast between the CRUTEM gridded data and the NOAA/GHCN gridded data for the Hawaii gridcell (157.5W, 22.5N). In one of his FOI obstructions, Phil Jones argued that CRUTEM data was already available at GHCN. But as you see, the CRUTEM gridded version for the Hawaii gridcell is remarkably different from the NOAA GHCN version.

To illustrate the stunning difference between the two series, here’s a plot of the difference. CRU has increased relative to GHCN by approximately 2 deg C during the 20th century. ([snark] I guess that’s what the Met Office means by the “value added” product/ [/snark])

Figure 2, Difference between CRUTEM and GHCN versions.

Reconciling these sorts of differences is quite time consuming and one would think that this would be the sort of thing that the temperature index providers would spend time doing. But unfortunately they seem to use funding for temperature indices to support other activities that they enjoy more and spend negligible time on QC – Gavin Schmidt justified the lack of QC on the basis that is unreasonable to expect NASA to be able to keep track of such things as they only allocate about 0.25 man-years annually to quality control.

I took a quick look at features of the underlying data to get some ideas. However, this sort of reconciliation is detailed and NOAA/NASA should either do it themselves or contract someone to do it for them.

All or virtually all of the Jones sites are airports. Six of 10 sites have name evidence of being an airport and are classified by GISS/GHCN as airports: Lihue WSO, Kaneohe Bay MCAS, Honolulu WSFO, Molokai A(irport), Kahului WSO and Hilo WSO. Barbers Point NAS (Naval Air Station) and Lanai City (see NWS here) seem to be airports, but are misclassified by GISS. (Gavin, are you on the job? Or did Harry exhaust you?) Hilo 912857 might be a version of Hilo WSO 912850, but the ID doesn’t tie directly to anything at GHCN. That leaves Lahaina as the only candidate for a non-airport site. Honolulu Observatory was not on the CRU list.

Many of these sites are lost to NASA/NOAA. To the extent that Jones statement that he uses GHCN is true, this means that they are also lost to CRU. Early ends occur, for example, in GISS data for Molokai Airport (ends 1981), Lanai City (ends 1970) and Lahaina (ends 1980). Again, these sites do not appear to be really “lost”. Lanai City weather is online here and Lahaina here.

As far as I can tell, the only current CRU sites in their Hawaii gridcell are Honolulu Airport, Lihue Airport and Hilo Airport.

But why is there such a difference between Phil Jones’ “value added” version and the NOAA GHCN version? Dunno.

It’s a secret.

Update: A further observation on the construction of these series. Both the CRU and GHCN versions prior to 1905 are based entirely on the series the current avatar of which is at Honolulu airport. GHCN applies an adjustment, presumably to reflect UHI (though I don’t for sure what they do.) It appears that Jones does no adjustment given the similarity between portions of his gridded series and unadjusted Honolulu airport.

I will be on Molokai on Tuesday for the day. I’ll take a look around. Don’t know how much time I will have at the airport. Maybe I’ll get a chance to check out Lahaina later in the week. There is a small airport for the Island of Lanai near Lanai city. Also a small airport near the town of Lahaina. Very few jets. Mostly small prop planes. But I guess we are really more concerned with data collection at this point. As Mr. Jankowski suggests, both would be great sites for the next PR Challenge, but may I also recommend the top of Mauna Kea for an inspection of the CO2 measurement facilities:)

If I may:
Steve; why don’t you apply for money to create your own temperature index? Use GHCN data and make another, lets say, independent contribution to the garden variety of station-based indexes that already exist.
In all honesty, getting an open source, fully archived and fully transparent index would be scientifically and technically highly interesting, and who would be more up for that task other than you?
I am not kidding. Such an index would also force openness on more established but less transparent indexes. Is this a good, decent or bad idea?

Re: Sune (#7), In principal it sounds good, but I wouldn’t expect someone to expend the immense effort to do so without some sort of compensation. Phil Jones is paid to do what he does. One ought to be able to reasonably expect that everything would be hunky dory, and yet it doesn’t seem so. This is troubling no matter what you believe…

Re: Andrew (#8),Andrew: That is why I proposed to apply for money to do the new open source temperature index.

Gary: Of all the data archived so nicely here, you say that Steve does not collect data?? How can anyone do statistical analysis without data?

I would not content that 1) an open source temperature index would not be used by any climate scientist – please remember that not all are Team members and 2) How says that such an independent index would disagree substantially with other indexes? We cannot know that in advance.

In fact, that is why I think much of the effort here is wasted time. The criticism of CRU, GISS and others is diluted in post after post. Steve is rightfully criticizing various aspects, but at some point you must also come to the conclusion that “if they can’t make a proper index/method and if we always know what they do wrong, then lets make our own open source index”. It is the ultimate criticism to make a method that can be completely fact checked. Both strategies (the current) and this one, is scientific in nature and it is just a matter of choosing the one with the greatest scientific interest. It is not politics please keep that in mind.

Gary: Of all the data archived so nicely here, you say that Steve does not collect data?? How can anyone do statistical analysis without data?

I should be more precise: By “collect” I mean that Steve doesn’t generate source data from instrumentation and field samples. Of course he “collects” source and processed data from primary sources into the file directories on the CA server.

It is the ultimate criticism to make a method that can be completely fact checked. Both strategies (the current) and this one, is scientific in nature and it is just a matter of choosing the one with the greatest scientific interest. It is not politics please keep that in mind.

Not politics? Actually, it’s as much economics and politics as science. In the ideal situation, a free-market of datasets, the “best” might win out eventually, but here the costs are prohibitive for later entrants. These costs are the expenses of gathering, cleaning, and storing the data as well as the effort to convince people who are suspicious of you that your data are “better”. It’s cheaper to fix the original dataset. The data ultimately are used for policy formation, which is politics, not science. To think it ends with determining the facts about climate is to miss the intentions of the people who run the world.

I should be more precise: By “collect” I mean that Steve doesn’t generate source data from instrumentation and field samples.

Some replies doen’t make a whole lot of sense as the quote above visibly shows. This post was about data (I repeat; “data”) from Honolulu (one airport station and one rural). Perhaps you also missed that the figure for this post shows GHCN data for a particular station. Call it what you want, but I see station data analyzed on many occasions here. I think we should leave it here. Because it can be a long debate if you are missing these basic facts and we are wasting space for others. Ok?

Re: Sune (#24),
As I’ve said previously, I don’t have the faintest interest in purporting to make a temperature index nor do I have the staff and resources to do so. In my opinion, these indices should be prepared by proper statistical agencies.

Re: Steve McIntyre (#31),
Is that it??
You don’t have the faintest interest in purporting to make a temperature index? You certainly ask enough questions to the maintainers of other indexes – quite frequently. But I do know this is only a hobby of yours and that we should not regard this blog as anything more than that. And I am the least to want to spoil the fun of poking at the big ones in post after post that never make a lasting impact. Please forgive me for suggesting the constructive and direct way forward that would elevate you hard work from a hobby to something that had to be seriously reckoned with. My apologies.

Re: Sune (#24), Except that money is already devoted to these efforts…why can’t those already being paid get it right? And why shell out twice as much just to find out whether you needed to or not? Maybe some have an appetite for this, but I doubt the pols do.

Re: Andrew (#32),
I actually laughed a little and I have a hard time giving you an intelligent response on your comment – but it was really fun to read – thanks Andrew :-)
Yes, money is going into CRU, Phil Jones, GISS, NOAA and what have you. And yes, Steve and others are criticizing them all the time. I don’t think the world would run more dry from liquidity if some were used to maintain an open source temperature index. I don’t really think you meant what you wrote about shelling out more money on this. Competition is good – or… shall we also debate this? Please don’t – the conversation is weird enough as it is already.

Wow – I feel kind of dizzy – I have to get back in the world of logical thinking and straightforward targeting of problems – bye guys.

I don’t understand your complaint. Suppose that I were an analyst for a brokerage firm and I wrote an analysis report that said that Lehmann Bros had a lousy business plan and that you should get out of the stock.

No one would say – set up your own investment bank if you’re so smart or else shut up.

EVen more ludicrous would be the suggestion that the analyst become an investment banker on his own without the resources or staff to do it properly.

Re: Steve McIntyre (#39),
Be it as it may. I am glad to have shaken opinions and I think I hit a sour nerve.
As I said. Much time is spent on criticizing CRU, GISS and others – rightfully so – nobody can disagree with the necessity of that aim. All I am saying is that a thorough criticism of temperature indexes would be to do the same thing you and Ross McKitric (and two committees) did on the Mann papers:Rework the problem from scratch.

When you did that it moved something – actually – is shocked the world of climate science and a few politicians as well probably. Poking at CRU, GISS etc. changes nothing and is at best a waste of time. That may be a pessimistic point of view, but it might also be a realistic point of view I’m afraid.

It was powerful to see how Mann’s work could be rejected by a detailed and complete analysis of the work. I am saying that the same needs to be done to the problem of temperature indexes before we can trust the criticism here as well.

I thought it is amazing that so much resistance to such an idea is generated here – the analogy with investments banks and analyst was cute. Again: Real helpful criticism demands a full rework of the temperature indexes and it would be a good project to do. The pure dismissal and lack of proposals on how such a big task may be approached was unprofessional and laughable to say the least considering what is done here regularly. End of story.

You may well think that a full rework of the temperature index is a “good project”. Fair enough, go to it. I said that it’s not a project that interests me.

There are a few statistical issues in the area that interest me – primarily procedures for dealing with inhomogeneity and their knock-on effect on adjustment procedures.

In order to do an index from scratch to the standards that I would expect from others would require a careful examination and survey of metadata that is well outside the time and resources that are available to me.

In the Mann case, we were able to examine statistical issues on a given data set. And recall that even in that case, we refrained from offering an alternative view of the history.

Re: Steve McIntyre (#62),
Fair enough. I understand that it is a problem of both interests, priorities and resources. The indexes have major impact on climate science and auditing them should be a high priority. Especially if you can find errors, mishaps and secret value-adding in CRU all the time. Auditing CRU and others is important. No one in their right mind can say this is a low-priority issue when we regard the impact these indexes have.

Why don’t you do the job you so lustily promote for others? There are people all over the world showing how, here and there, the numbers do not add up. I’ve just spent 12 hours doing that for a small portion of the globe, because I was lucky enough to have access to early data that have been officially “vaporised” by now. I thought I’d find a change that made old temperatures lower and recent temperatures higher, and I did. There’s an adjustment difference of about 1.5 deg C over 100+ years. Part is the maths of manipulation, part could be because I have slightly different data subset.

Compare this with the fuss being made over a 0.8 deg global change in the IPCC estimates for the 20th century.

You have not hit a sour nerve. You’ve injected intellectual laziness into the busy lives of others, who are scratching to find data to start from scratch. Strange how hard it is to find.

Re: Sune (#61),
I’m sorry we have failed to communicate clearly with you. You’ve not shaken any opinions nor hit sour nerves that I can tell. You simply do not seem to catch the huge, pragmatic difference between what is currently being done by the “skeptic” community and what you suggest.

It’s one thing to poke at statistical analysis and sometimes rework it from scratch. That’s most of what Steve does, now joined by the Jeff’s, Ryan, etc. Very valid, very pertinent since so much climate change research falls entirely within this realm.

It’s quite another to create a new ongoing data stream as you suggest. There’s absolutely no reason to go to that effort once without setting up infrastructure to keep it going. And that requires infrastructure. Note that the only field data/index-related “skeptic” work to date has been the Almagre adventure, and that (as already noted) was mostly to prove the Starbucks Hypothesis, and partly to generate a small verification dataset.

Somewhere in between is poking at the metadata, which is what you find at SurfaceStations — a better one-time snapshot (so to speak :-)) of current metadata is itself a huge undertaking.

As to “dismissal and lack of proposals”, please look around a bit more. You’ll find plenty of material discussing what it takes to do a good job of data collection and indexing. Dan Hughes is one who has a hot button for this. You’ll find a link to his blog on the left hand side of this page… and lots of archived discussion here.

I agree. Steve, Anthony, et. Al. should invent a time machine and remeasure the historical data. Nobody should criticize temperature reconstructions unless / until this problem is reworked from scratch.

Otherwise, we won’t be able to “trust the criticism” of a person who finds holes in Phil Jones’ work.

Steve and company should immediately cease attempting to procure Jones’ publicly-funded data. They need to go get a DeLorean, a lightning rod, and a flux capacitor.

Re: jim edwards (#66),
Amazing ignorant. It has clearly never been the intention to stop anyone in their quest for Phil Jones secret data. That was not the idea, nor part of the argument. Of course Phil Jones should hand over his data. The problem is that you have to have a complete result of something being suspecious or down-right wrong. Getting Phil’s data is certainly a step on the way. Creating a fact checked, no BS index from publicly available data (NOAA) is the best and most convincing way of telling if CRU got it wrong. An open temperature index based on correct and tested methods would make it clear to the community and the public how much Phil Jones “value-add” to the raw measurements. Don’t quit your day job please.

I’m sure you’re correct. Nancy Pelosi can’t wait to go out of her way to re-open the debate that ended ten years ago and provide funding to statisticians Joe Schmoe and Don Quijote, so they can “completely” duplicate the temperature work that the consensus of scientists believes is correct.

If Schmoe and Quijote come up with a contradictory result to the one relied on by IPCC and all reputable scientists, I’m sure that Obama, Gore, Pelosi, Boxer, and Waxman will immediately call for a halt to cap-and-trade legislation and proposed regulations on CO2 emissions.

I’m curious why Steve or anyone else should do the work of Jones when Jones is clearly qualified to do it? All that’s really being asked is how Jones uses his data. Having anyone else go through the effort of duplicating Jones work just to have another index is defeating the purpose of having Jones just tell us how he comes up with his data. After all, policy is being made using this data, and right or wrong, shouldn’t we know how Jones came up with the numbers?

Re: Sune (#6), Steve tests statistical methods and doesn’t do data collection. Even the expedition to update the bristlecone chronologies was mostly a test of the Starbucks Hypothesis. Even if a new pristine temperature dataset was compiled, the majority of climate scientists would reject it because so much of their work has used the existing versions. And if a new dataset disagreed substantially with them, which do you think they would favor anyway?

Ok, I’m starting to get used to this stuff. Why, is beyond explanation. I hope Gavin is paying attention.

Alright, I’m still surprised daily — Fn, Wow!!

I really am going to have to snip myself — a lot. Has anyone ever seen a decreasing temp station accepted or are those tossed? I’m just a dirty fingernail engineer but the quality of this data cannot discern a trend of 1.5 C/century. No statistical proof, it’s just an opinion but – it ain’t gonna work guys.

I had a post on the Antarctic pointed out to me. Drs. Steig and Schneider gave an interview on Antarctic warming. The article was self contradictory as I read it.

Here is a dataset from either Warwick Hughes or P Jones showing an increasing temperature. Compiled about 1990, so I do not know how much added value. It shows the mean annual temperature of 6 of the larger cities in Australia. I believe it was used in part or whole in early CRU warmist papers.

Here is a dataset from the same origins. It shows 25 small or rural stations. I understand that it was not used in the same way as the dataset above.

However, you asked for an example of a falling temperature graph, so here is one.

The graphs were complied to support an emerging UHI argument and presented as “Tasman Institute 1991 review of the Australian component of temperature records used in the 1986 Jones et al Southern Hemisphere paper”, (ref at top of URL cited.). At the time, I was an industry representative assisting the Tasman Institute to choose projects deserving of analysis. The Tasman Institute was a private think tank with some exceptional staff.

It would be interesting to recreate the graphs from present day global sources like GISS and HadCRU and KNMI. It might (repeat “might”) reveal an example of selective data use. P.D. Jones once emailed me that he had lost the data used in this 1986 paper.

Re: Geoff Sherrington (#28),
Here is a graph of the Giss station data for the capital cities with the Jones graph super-imposed.

Note the yellow arrow in the early record. It’s an example why anomalies work out better than averaging the temperature over time at multiple stations. Compare the same year to the bottom graph which is done as anomalies.

Five stations reported annual temperature in 1891 and 1893. Only four reported in 1892. Hobart missed a year. It is the coldest station, averaging two to seven degrees colder than the others. That causes the large jump in 1892 when averaging by temperature. About 0.9C difference since its annual temperature wasn’t reported. Don’t know where Jones got data for Hobart that year. Maybe a private dataset or winging an estimate.

There are some inconsistencies between the Giss and Jones data, but it really starts diverging from Giss around 1960. Why I don’t know. I used all the Giss datasets available when creating the graph. 13 datasets across the six stations with each station having at least two datasets. I’m sure that linear trend line would be a couple tenths steeper with the Jones graph than it is with the anomalies. It’s 0.44C the way it is.

Bob, I was not worrying so much about the capital cities graph because it already showed a rise, so why bother to add more? I appreciate the work you have put in. Thank you.

I’m in the final stages of checking the GISS homogenised versus Warwick Hughes data for the 25 rural stations. In preliminary view, it’s a rather different outcome. I’m double checking before posting because if the subset is representative of the whole, it has significance – and a possible explanation rather like yours.

Just now I’m in the middle of seeing if I can find current versions of all of the mentioned stations to recreate a post-adjustment graph. The original data comes via this link
Re: Geoff Sherrington (#28),

I compared data and reported on substantial differences. I did not opine on which is “correct” nor do I see how I could, given the data at hand.

In any event, I have no opinion on which is “correct” nor do I think that it is reasonable for you to expect me to have an opinion. Surely your question should be directed to Hansen or Jones (and I’d be interested in their answer.)

The CRUTEM version is equal to unadjusted Honolulu for its early portion. The GHCN version is also based 100% on Honolulu in the early portion – you can prove this because the delta series is flat over this portion. However, the GHCN version has been adjusted for UHI(??) while CRUTEM hasn’t.

The HadCRU land-plus-sea version is different from either and at a first glance does not have a steep upslope.

The land data is less important than the SST data – which makes the lack of critical attention to SST disappointing – but problems with land data will also affect “Attribution” studies where land vs ocean is held to be a “fingerprint”.

Thx. it’s off topic but whenever Islands and CRU pop up I just recall an interesting unfinished discussion we all had ( RomanM and others) on the Jones approach to blending land/sea for gridcells that were coast or island.

“Another problem with gridded CRU data is that there is no adjustment for station elevation, and thus we have adjacent cells in the island of Hawaii with 5 decrees C difference on the 0.5 deg resolution data set:

The western cell may correspond to one of the high altitude stations, such as Bradshaw AAF at about 1900m. I wonder whether the apparent marked shift upwards at the beginning of 1986 is natural or not:

OT
Has anyone seen the paper in Science by Gunnar Myhre about the cooling effect of Aerosols? Apparently the cooling effect is at the weaker end of previous thinking. Wouldn’t this mean that CO2 sensitivity is also weaker?
.
According to the BBC report here: http://news.bbc.co.uk/1/hi/sci/tech/8108100.stm
“Global models of the emission of these aerosols suggest the cooling effect they have cancels out approximately 10% of the global warming caused by greenhouse gases,” explained Jim Haywood, an aerosol researcher from the UK Met Office, who was not involved in this study.

“But satellite methods that detect the amount of aerosols in the atmosphere suggest a cooling effect that cancels out about 20%.”

By identifying the source of this discrepancy, Dr Myhre was able to reconcile the two approaches and come up with a more precise estimate – closer to 10%. This suggests the effect is weaker than the Intergovernmental Panel on Climate Change (IPCC) has estimated.

Of course, naturally, I completely agree with S. McIntyre here. In a way, his WORK (not “hobby”) is akin to that of a movie critic, book critic, music critic, etc. His job is critiquing what’s out there, thereby (one hopes) helping us all to see more clearly, to develop better our own critical eyes, and–possibly, or at least conceivably–to provide an impetus for the “out there” to improve their product. (Would that “peer review” accomplished the same task!) And, a good movie maker, writer, etc. in fact welcomes the critiques: the more, the better! More chances to learn and grow. Both the creators and critiquers are valuable, and, as far as I know, only rarely is there one who chooses to engage in both to any extent (right now, all I can think of is T. S. Eliot, from my limited perspective).

I am interested in CRU data for some climate analysis (unrelated to whether the earth warms or cools), but unfortunately Sune’s insistence is deviating the attention. Fine you argue, but may be we can find an OT common place.

So back to CRU. CRU TS 2.1 and teh CRU the 10 minute grid has temperature data for the main island (Hawaii, where Mauna Loa is -the big volcano-), but I haven’t managed to get any corresponding coordinates for the elevation data, so I don’t know to what altitude corresponds that grid temperature.

Similarly, in the list of stations, I do not find any one within the island coordinates (-156.0 to -154.75 and 19.0 to 20.25).

am I not looking properly or there are a lot of stations which are not reported?

The GISS for example is more and more moving away from UAH/RSS if you look back from 2002. There will come a point where its clear to all that they cant all be showing the correct temperature development.
And in that situation perhaps UAH/RSS will be miscredited?
Therefore an independent ground based temperature record would be very helpful.

Re: Frank Lansner (#45),
I agree naturally. As long as there is a monopoly on the ground-based temperature index then we can jump and scream as much as we want. The UHA/RSS index cannot do the job alone. There is always the benefit of the doubt on the ground-based indexes. There are not properly challenged.

Whether it is demanding or not to make such and independent index is irrelevant – the thing is that the debate of who is drifting away from who cannot be resolved without someone breaking the monopoly on the ground-based indexes.

It is that kind of “to the point” thinking that is essential – so many thanks Frank. The doubt must be confronted because right now doubt is obscuring the science and the policy decisions which is based on the doubtful science.

Re: Kenneth Fritsch (#50),
I’ve used variations on that joke for years, but no one ever gets the reference.

I was thinking “come back soon” but “come back Shane” or “Shane, come back” as it was shouted in the original works for me. So: Come back Sune, Come back, Sune. Oh, I see your already back. Never mind.

Has anyone ever superimposed a graph of increased traffic at an airport on a graph of the temperature record from that airport? Air conditioning units and BBQs distort many records and it would seem likely that DC3s might have had less impact than 747s.

You’re mixing issues here. The main problems are not how to do things on a going forward basis. There’s lots of information on that from people far more familiar with practical measurement issues than I am.

The issues at hand here are the interpretation of historical data with imperfect metadata.

In order to do an “audit” as carried out in a business setting, you need to have the full attention of the person being audited and access to their records and files – and, of course, CRU keeps that sort of stuff secret and actively discourage people from being able to verify their methods and data.

Why stipulate ground level? Is it not about the noisiest place? The experts can’t even seem to agree if there is a temperature unconformity between air and land/sea. Some experiments with tall masts suggest a better location is a few hundred metres above the surface. Even the Mauna Loa CO2 are rather different when you go down to the island’s ground level near the sea. Let the atmosphere integrate and smooth for you.

Frank Lanser: People have thermometers at home, they will notice, as will people living in most areas of the earth lately, even if they support/believe in AGW. They are/will be noticing that there is no change. If anything most people in the USA, Europe and South America would be feeling/noticing/recording cooler temps during the past 2 years… that’s where eventually GISS, HADCRUT etc will fade away…? BTW the cooling is supporting the satellite data. So far… it looks like AMSU data is going to show a negative anomaly for June 2009.http://discover.itsc.uah.edu/amsutemps/ (see channel 05)

…which shows that climate models are way out on simulation of precipitation. This is important and not just in itself. Since the positive temperature feedback of the GCMs is based on the behaviour of vater vapour, it follows that if precipitation is incorrectly simulated so is water vapour as the associated feedback.

As a no axe to grind lurker for quite awhile, may I suggest that Rome is burning while the stats are fiddling. It takes more than re-creating someone’s data and methodology that they used to write a technical paper. There is lots of brainpower in ClimateAudit. Using it to tell Sune to not play in your sandbox is a poor use for the brain energy. Telling someone to do it themselves is simply poisoning the idea well. One never knows when or where a good idea will emerge.

It is well understood that Steve can only do so much by himself. Frankly I am always amazed at how much he can produce in such a short period of time. It takes time and effort to include the amount of backup detail used by Steve.

On the surface Jones raises my red flag as it does Steve’s and the rest of you. Let’s presuppose for a moment that Jones work will not pass stiff scrutiny. What is the best way to win the truth battle by falsifying the assumed bad work by Jones? Who are the target audiences? What is the best way to get him/her/their attention? What data is needed to demonstrate that Jones work is bad? What does it take to get the data? When should the hammer be dropped on Jones? What if Jones work is good, not bad? Etc.

If you have ever looked at US surface station records, the location and equipment change fairly frequently. There is an issue with when during the day the temperature is measured. There are siting issues. It sounds easy to start an “open source” temperature data source, but this would be very manpower intensive. One would have to have a good deal of time on their hands just to coordinate the effort, much less do the grunt work. So basically yes it’s a good idea; but who would have that kind of time to do it? I don’t blame Steve for not wanting to do it. Besides, it would detract from the great work he is doing on the stuff he enjoys.

I don’t think most of the persons suggesting this “great opportunity” have any real idea just what would be involved in setting up and operating such an enterprise.

Start be setting up the infrastructure and contacts with the existing sources of temperature measurement. This includes acquiring all of the background information on stations not only in the US, but in the rest of the world. Next update all of the population data and make sure that classifications of stations (rural, urban, airport, etc.) are correct. Check contradictory sources. Write all sorts of software. Acquire raw data. Institute quality control checks. Every month chase down the new temperatures, etc., etc., …!

At the end you have what you think you have a good product. Convince everyone that it is “better” than CRU or GISS (of course, in the mean while you figured out what CRU does so you know what wrong with their stuff). All in short order, so the rest of the time you can continuing chasing down all the new junk coming out from the Climate Science machine. Piece of cake!

My suspicion is that Sune has to be an agent sent by Gavin to lure Steve away from where he can be an effective thorn in their side. ;)

Re: RomanM (#78), I don’t think that there is a chance in hell that Sune is going to lure Steve into anything, much less what he seems to be suggesting. Come to think of it, what exactly is he suggesting? How about it, Sune, Kerkamp, lay it out. Exactly what you want done and exactly how it is to be done. Then give us an estimate of the costs, manpower, time, etc.

Trying to replicate/duplicate GISS/Hadley indices with all of their quirky adjustments would be a major undertaking. What would you think, however, of doing a rigorous, totally transparent, manageably small QC check of their global results? A few score well-situated stations might suffice for the purpose.

But that IS what has been going on for the most part with GISS in the past. No, it wasn’t randomly selecting random grid squares, but in watching the behaviour of the global record and noticing aberrant results. The year 2000 problem leading to a readjustment of the “warmest” US years was also such a result. Individual stations have been analyzed (often with the cooperation of the WUWT blog with some success in uncovering errors.

None of this is really possible with the Hadley record because of their refusal to disclose the details of any of their methods. They have a paper which discusses their methodology from a theoretical perspective (and contains some spurious thinking on combining land and sea within a grid), but is of no practical use in deciphering their database or the adjustments within it. The whole purpose of this thread is to try to pry some of this information out of the void in which it has been ensconced.

It took some time with GISS and it will likely take somewhat longer with Hadley, but my feeling is that the information will eventually be available.

Re: RomanM (#78),
There are many success stories from the open source community. Linux, Mozilla/FireFox, StarOffice etc. These are all big and extremely complex software products that were build (at least initially) from the hard work of thousands of volunteers with time on their hand and filled with enthusiasm of creating something great and with an important and lasting impact.

But the same strategy could be applied for an open source temperature index. Establish a steering comity and have a process where volunteers can be accepted if they can be checked and verified. Then let the broader community do the hard work based on properly defined guidelines. Have internal review for QA based on sampling statistics so the work done is ensured to be valid. You be surprised as to how much can be accomplished in that way – for free.

The bit about me being an agent for Gavin is at your own expense of course. You small tricks and FOI have no real impact. See how little the defusing of Mann has made in the climate science arrena. Mann is even successful at publishing false response letters in PNAS.

You may have the facts and you may be right – but have it ever mattered? Have it ever really stopped wrong or suspect papers from keeping emerging with all the bad impact is has on how humanity view the world? Is what you do now likely to change anything? I think we all know the sad answer to that little question. Think bigger, you should give it a try sometime – its a cool thing.

A comparison of the open source community with a one-trick pony such as a temperature record is spurious. Getting it started and making it operate is an enormous task completely out of proportion of any possible positive benefit to those of us who think that there is too much exageration in the propaganda efforts of the hardline AGWers. IMHO, it’s a nonstarter from any realistic cosideration.

I won’t speak for others, but I think that my own skills can be better applied in other venues of the debate.

Having said that, Jones’ obstruction is a disgrace as is the acquiescence in his obstruction by the Community. Every day that this goes on, Jones embarrasses the “Community” and does a disservice to the cause that he’s advocating.

As I’ve said before, readers should not expect that there’s some smoking gun in the secret data set. I don’t expect one. My own opinion, as I’ve said before, is that Jones doesn’t want to reveal the negligible amount of quality control that they do.

Gavin Schmidt says that GISS spends 0.25 man-years on their index. CRU probably does the same. They probably have old, old programs that like GISS, sort of work, but they’d be embarrassed to show anyone. And for running these ancient programs once a month on data prepared by others, they get lots of money.

Once their data and methodology is disclosed, my guess is that it wouldn’t take me very long to put a CRU emulator online and call into question why they get $1 million per year or whatever to calculate their index.

I think that the obstruction is monetary and has nothing to do with climate change. (I could be wrong.) HAving said that, the Community should not tolerate Jones’ behavior any longer.

Re: RomanM (#102),
I can’t see that using GHCN data as provided by NOAA and make an index with the proper adjustments (or perhaps lack of adjustment) is such a difficult job. In addition, the open source strategy is certainly able to lift the task and I have not seen any reasonable argument as to why it would be an impossible task. Basically we disagree about one thing: Is criticism as effective as providing alternative measures when we keep finding problems in the existing ones (CRU etc.).

I say that given the highly successful progress of AGW promoters in resent years, both in terms of the huge impact it has had on science and various political decisions, one must seriously question the current approach of being critical only.

In fact, the continued and accelerating success of AGW proponents is the result we need to test and, in this case, reject the hypothesis you favor, namely whether your approach of directing well-founded and pinpointed criticism is able to debunk wrong and false science (Mannian science for instance). The result shows that wrong science keep being referred to (Mann etc.) while the critical (and better science) live a remarkable quite life gathering very limited attention.

That strategy clearly has it deficiencies and thats why providing better alternatives to e.g. temperature indexes is interesting because it may have a more immediate and direct impact. We can disagree about many things, but one important thing is quite evident: The current strategy is only interesting for those who criticize. The wrongdoers are immune to your attacks unfortunately. They thrive more now than ever and seem to grow in strength with every attack. That is a shame and new approaches need to be explored – at least intellectually.

The result shows that wrong science keep being referred to (Mann etc.) while the critical (and better science) live a remarkable quite life gathering very limited attention.

The same would simply happen for what you are proposing. If the results show a result contrary to the “expected” result, it would be accused of bias because it is run by volunteer “sceptics”, not even an “official” department, and summarily ignored. What defense could you use against such an accusation?
.
At least what Steve does cannot be ignored as easily because his reasoning and results are fully open to verification and scrutiny. Of course, he cannot force people who have already chosen their “side” to reconsider the evidence. Noone can. People will still believe what they want to believe and choose their sources based on their world view.

Re: DaveJR (#110),
Stay cool and just read before rushing to an answer. I have cluttered this thread with numerous comments explicitly emphasizing creating “an open source temperature index”. Ok? That means that all procedures, incoming raw GHCN data would be available. Probably written in R or even reproduced in Matlab or Python or whatever programing language you prefer.

The idea is that its completely open. It should be downloadable so everybody can keep a version on their laptop. All scripts and methods should be clearly documented an open to criticism. We would want criticism because it is part of getting out the message. These here is a competition and the skeptical or critical side represented by e.g. Steve must start to really compete. This issue is to tampered with to trust that the scientific field can resolve this in a civilized matter. Believing that is a dead end strategy and a slow death of healthy decent.

People tend to trust things they can see and play around with. An alternative index accompanied with complete openness provides that to climate science when needed.

Sune et al
/on soapbox
My present understanding and opinion is the satellite data is the new “open” data source for air temperature. I have not read any accounts of Dr. Spencer hiding his algorithms or data.
It has coverage that you can’t possibly get with a ground based network. Ground based stations with the long term accuracy, stability, and durability are very expensive – and will always be subjected to changes in the local environment. You are also dealing with thousands of ground based sensors to monitor and maintain versus a satellite (or a dozen) with one sensor to maintain which is used for land and ocean temps. Whatever error band the satellite has is consistent for all measurements – unlike the ground/sea based network.

There are – as extremely well documented on this site – issues with proxy based reconstructions of past temperature as well as the present ground/ocean network. While it is “fun” to consider what climate was like in the past thousand years, it appears to me that the reconstructions do not have the accuracy or resolution required for more than general statements. Trying to extract more precision out of them with statistical algorithms is something I do not believe is possible. If “smart” folks with Phd’s can only get “reasonably” looking global temperature reconstructions by picking out a particular tree ring on a mountain range in the western USA and giving it several hundred times the weight of the other proxies in the reconstruction (and that is just plain silly IMHO), then I don’t think it can be done. While Steve has never said it is not possible, he has pointed out the flaws in the reconstructions used by IPCC.

The present ground network was not designed for the the purpose of monitoring climate change. The stations do well what they were designed to do – monitor local weather conditions for (mostly) agricultural purposes – not climate. Again – trying to extract a small climate signal out of them by creating models to “correct” the readings is not consistent or reasonable with their purpose and design. I think it is wishful “thinking” to claim that a small climate signal can be extracted with precision. Too many variables – too little measurement precision. Again, Steve has never claimed it cannot be done, but he also has never claimed that it is possible, either.
My opinion.
/off soapbox

In a general way, the problems with the widely followed indices are well known. I had in mind something more formal: a small-sample compilation of roughly equivalent indices (geographically representative, but not necessarily gridded), using records only from reasonably untainted surface stations. The same statistical techniques that might detect election-result irregularities in a demographically inhomogeneous population could be applied.

As far as HADCRUT3 is concerned, I never put any trust in it for the simple reason that enough data (prior to satellites)for constructing even yearly-average time series isn’t available in most Marsden Squares outside of well-traveled sea lanes. “How’d they’d do that? is what needs to be asked. The embarrasment of answering that question is what might be holding up Hadley’s disclosure.

The problem is in finding something which might prove to be an effective evaluation which could reasonably clearly point out some of the shortcomings of the methods and/or the results. All humour aside, if you have some specific practical ideas for implementing your sugestion, I would like to hear them.

Re: RomanM (#85)
With GHCN data widely available, it’s no big deal to run small-sample validation tests on the popular long-term station indices, avoiding obviously tainted records. Geoff Sherrington (#23) has shown some telling results along those lines for Australia. I’ve performed penetrating tests on a much wider basis, for which I’m handsomely rewarded. The results are remarkably consistent in revealing index bias. There’s no reward apparent, however, for providing a blueprint for professional testing to an audience intent on dismissive humour.

There’s no reward apparent, however, for providing a blueprint for professional testing to an audience intent on dismissive humour.

I don’t know if this is a reference to my comment #85 or not. If it is, then you misunderstood what I said. My phrase “all humour aside” was my backhand way os saying “in all seriousness”. I would like to see a practical blueprint or a more detailed description (or better still the results of an implemented example or two) of what you are suggesting.

I don’t see this as a simple straightforward matter particularly in regard to the final comparison of the results of such an analysis. For example, on what basis can you justify that your results have identified “index bias” in a process whose particulars have not been disclosed when there is no absolute standard for the comparison?

Re: RomanM (#96),
There was no problem in understanding the phrase you explain. I simply failed to find much truly humorous purpose in your preceding sentence.

On the substantive point, I disagree. Complete records from stations in temporally stable (usually non-urban) environments provide an excellent standard for comparison. Procedural verification is not the same as validation. The anomaly mincing machines of GISS and Hadley don’t have to be replicated for validation of results. Even Gavin at GISS concedes that their index can be essentially obtained by averaging 60 station records, though I doubt he had untainted ones in mind.

When independent small samples consistently show one thing, and the popular indices quite another, systemic bias is the maximum likelihood conclusion in a Bayesian context. You’ve already seen Geoff Sherrington’s raw comparisons. I’m not at liberty to disclose the details of my employer’s industrial-strength methods or the results.

Re: Sune
I get it. I do not think you do. I have quite a bit of experience on the boundry between the material science world and the application of materials to products. Nothing to do with climate studies but lots of money can be involved. I now know from experience that when a new advancement is announced but one must sign an NDA and pay money to get samples, when the advancement cannot be independently confirmed because of the lack of access to data, when letters and numbers are used in published papers to represent secret compositions, watch your wallet! Those “scientists” care about some goal other than getting the science right for the benefit of society. This can only happen, and it does all too often, because those managing the technical journals or those selecting presenters at conferences allow it to happen. Eventually, the truth wills out because Mother Nature will not be denied. But, that takes time. This situation I described happens in my little corner of “the pure pursuit of science” but I am sure my story sounds familiar to the CA community. Steve’s site is deja vu all over again!

To put it simply, I did not have to understand all of the mathematics behind his analyses to recognize a “bad science” warning the first time I stumbled across his web site. Correcting the numbers isn’t the point. Getting the science right is. My suspiscions are now aroused. How about yours?

I am very concerned that Phil Jones of HadCRU (which as I am sure you know is the Hadley Centre Climate Research Unit) has been consistently refusing to release source data to those who wish to review it.

As a matter of basic scientific practice the source data and methodologies should be published with the research findings.

Since this research is publicly funded, the case for release is even stronger

Since significant public policy is being based on this research, it is vital that the data is available for validation and assessment by others.

Steve McIntyre is no friend to the Global Warming cause so the failure to release data can only damage the credibility of the case for urgent political action on GW.

But the failure to provide full release is in my view shocking in any case.
I ask you to use your influence to bring this sorry and damaging situation to a proper resolution.

My best wishes Sir,

Stephen Haxby

To which reply – (23/06)

Dear Mr Haxby

On 3 October 2008 the Prime Minister announced the creation of a new Department of Energy and Climate Change (DECC). The new department brings together much of Defra?s existing climate change responsibilities with the energy component from the Department for Business Enterprise & Regulatory Reform, to focus on solving the challenges of climate change and energy supply. Defra is currently responding to letters on these issues on behalf of DECC.

Thank you for your recent email about climate change data. I have been asked to reply.

I should first explain that there is no such organisation called ?the Hadley Centre Climate Research Unit?. Professor Jones is director of the Climatic Research Unit (CRU) which is an organisation affiliated to the University of East Anglia. HadCRU is the generic name of the global mean surface temperature time-series produced by the Met Office Hadley Centre (MOHC) from the independently produced CRU land dataset and a global sea surface temperature dataset maintained by MOHC. The current version is HadCRUT3.

Although I accept that you are understandably concerned over this issue relating to scientific practice, the CRU is an independent organisation which receives no DECC funding for developing the CRU land dataset and therefore DECC does not have any proprietary rights to this data. It is up to Professor Jones, as the dataset?s owner, to release this data. So far, in response to various freedom of information requests, he has released only the names of the meteorological stations used to compile his dataset, but the station data for many of these (though admittedly not all) can in fact be obtained at the Goddard Institute for Space Studies (GISS) website at http://data.giss.nasa.gov/gistemp/station_data/ .

I should further explain that in our view this issue has little bearing on the scientific evidence that the world has warmed significantly over the past hundred years and therefore on the Government?s climate change policy which takes account of this evidence. Apart from the fact that Professor Jones has published his methods in scientific journals (see the Climatic Research Unit website at http://www.cru.uea.ac.uk/cru/data/temperature/ ), the HadCRU global temperature graph was one of four that were cited by the Intergovernmental Panel on Climate Change (IPCC) in their 2007 Assessment as evidence of the warming that has occurred since the end of the nineteenth century. One of these graphs was produced by GISS who do make available on their website all the station names and the associated temperature data along with their calculation computer code. The graph produced by GISS is very similar to HadCRU (and the other two independently produced graphs) but is calculated by a different method to that developed by Professor Jones. The close similarity of these graphs indicates that there are no concerns over the integrity of the HadCRU global temperature graph.

Re: SH (#88),
The interesting thing about this letter is the assertion that Phil Jones owns the data.

The University of East Anglia has their Intellectual Property policy online , and it states that aside from scholarly work (which does not include datasets or computer code), they originator is obligated to make a disclosure to the university so that they may properly protect and exploit the IP. (Similar to most institutions, I would gather).

Given that Phil Jones shares his dataset with some, and not others, I wonder if:

1) He has disclosed the dataset to the University
2) The University has protected the IP according to their own policies (meaning, is PJ following their policies, or making up his own? Technology Managers tend to feel very possessive about possible income sources.)
3) Are the data sharing arrangements approved of by the University’s technology manager, or is PJ simply acting on his own. (Effectively converting University assets to personal assets).
4) Whether he has maintained “strict confidence in relation to such Intellectual Property in order not to jeopardise the validity of any form of protection which might be sought”

Might be interesting at some point to put a bug in the Technology Manager’s ear, along the lines of “I would like to license a copy of Phil Jones working dataset and code. I understand that he has contracted to share this code with other institutions. Is there a set rate?” And see where this gets you.

There seems to point in replying to this response whose fatuity advertises its intransigence. To say we have confidence in the data because it agrees with another set of data is mind blowing.

For those interested, a little history. The current Secretary of State for Global Thermostats is Hilary Benn. He is the son of Tony Benn, AKA Anthony Wedgewood Benn, who renounced his peerage in order to serve in the 1966 – 1970 and 1974 – 1979 Labour cabinets. Father and son are both transparently clever, charming, and utterly bonkers. Tony Benn now serves as a national treasure on the lecture circuit and has for many years, it is said, drunk up to 6 pints of tea a day. It would have been eccentric, even in 1940 or so, so have named your son Hilary, and this now serves as the inspiration for some satire.

A parenthetical quote from your blog: “Gavin, are you on the job? Or did Harry exhaust you?”

Steve, while I’ve usually discouraged adversarial riposts as counterproductive to constructive dialogue, sometimes one can’t help but laugh. And I suppose as an initial condition, a modicum of dialogue is first required. I’d still encourage avoiding such in correspondence, of course. And perhaps you should provide trackbacks for those who aren’t familiar enough with the history to get the (inside) joke =).

We have asked Phil Jones for data so that we could compare the synthesized surface temperature with actual station data. Jones has provided everything that we have asked for. This is for our study of the 1930/40 climate bump that is ongoing. Alas, these things take time. But my experience has been quite different to yours.

As you know, I have often complained that the right wing and the left wing (the absolutists of AGHW and those who do not have a bar of it) have forced us into corners in which we are not comfortable. If there is to be reasonable resolution of the climate GWH issues and the fidelity of data (both critical and reasonable questions?) I think that the questions and opinions can’t be shouted from one corner or the other.

BTW, we have a Science article coming out next week about the changes in form of El Nino (GHW or natural variability: no idea! But changes there are) and its impact on NATL hurricanes. Not sure if it will be of interest to C-A as it does not raise the question of GW. But the data set is short……..

Re: Peter Webster (#100), as always it is good to hear from you. I have tried to get data from Phil Jones by requesting it, and failed. I have tried to get data from Phil Jones with a Freedom of Information request, and failed. I asked him once again politely, and failed again.
.
And yet you get it from him … could you tell us your secret, please? Because with me, he’s all like “Confidentiality prevents me from sending it and besides, the margin of my email is too small to contain it” … say what?
.
Also, I’m not sure to what you are referring when you say:
.

As you know, I have often complained that the right wing and the left wing (the absolutists of AGHW and those who do not have a bar of it) have forced us into corners in which we are not comfortable. If there is to be reasonable resolution of the climate GWH issues and the fidelity of data (both critical and reasonable questions?) I think that the questions and opinions can’t be shouted from one corner or the other.

.
Steve M., and I, and Warwick Hughes, have all tried reasonable requests to Mr. Jones. They were not shouted, they were not shrill, they did not force Phil into a corner — they were simple requests.
.
But even if they were shouted from the far corner as you describe … so what? Do we have to say “Mother may I”, like in the child’s game, to get some deliberately hidden data paid for by your and my taxes? I thought science was above children’s games. You say “if there is to be reasonable resolution” of the issues, that we can’t be shouting from one corner to the other, which makes sense … but you are totally ignoring the fact that until Phil ceases his childish actions and releases the data, there cannot be any resolution, reasonable or otherwise, in corners or not. I don’t care if you want to claim I’m forcing you into a corner, that’s your business. But claiming that the problem is that we’ve forced each other into different corners is absolute nonsense. The problem is that Phil Jones is refusing to release his data, plain and simple, and all of your honeyed words can’t change that.
.
In any case, could you please reveal your secret, your coded phrase that gets you past the Jonesian Guard to the secret data stash? I’m getting tired of beating my head against the wall. Or did he just give it to you because you are friends? Did he put any restrictions on your re-distribution of the data?
.
Finally, I have to say that for evil to triumph, it is only necessary for good scientists to get the data through backhand channels and never, never say a bad word about so-called “scientists” who hide and conceal their data.

The problem is not that people have been “forced” into corners as you claim. It is that your friend Phil has hidden the data in the most recondite corner he can find, where he and you and his special pals go and share it away from the harsh light of reality and the ugly gaze of unbelievers. The only rules for the sharing seem to be, never discuss the palpable dishonesty of the process, and never say a word about people masquerading as scientists who are hiding publicly-funded data.

-snip-
.
Truly, I don’t get why anyone in the climate science community puts up with this bullshit. Is everyone afraid to say something? Is there a secret oath I’m unaware of? Why have only a tiny number of climate scientists such as yourself stood up to call bullshit on this nonsense?

Please RomanM (#102) and Sune (102), please read my post. It may be inconvenient to address but if you are open minded…… Unless you want to stay in the “red” corner” or the “white” and not advance. Tiring I know but one has to address one’s precepts. I like to think I don’t have any but maybe I fool myself!

Pursuant to the Environmental Information Regulations, I hereby request a copy of any digital version of the CRUTEM station data set that has been sent from CRU to Peter Webster and/or any other person at Georgia Tech between January 1, 2007 and Jun 25, 2009.

Thank you for your attention,

Stephen McIntyre

IF this data set’s been sent to Peter, then they are going to have to be very inventive in thinking up reasons for it to be confidential under the legislation. They’ re running out of wiggle room.

Dear
Pursuant to the Environmental Information Regulations, I hereby request a copy of any digital version of the CRUTEM station data set that has been sent from CRU to Peter Webster and/or any other person at Georgia Tech between January 1, 2007 and Jun 25, 2009.

Wouldn’t it have been better to have asked for all CRUTEM station data sent from CRU to any person or organization outside of CRU including but not limited to Peter Webster and/or any other person at Georgia Tech between January 1, 2007 and Jun 25, 2009.

I have worked with lawyers in litigation and have picked up some of the ways that they think. The above wording would be what I would expect to come from them.

I don’t have a magic decoder ring (or at least I don’t think I do). I just asked for some specific data and received it. I am a firm believer of open data. there can be no other way. years ago, we had scientists going out in the field writing papers and holding onto their data. As part of the TOGA experiment (big El Nino thingo in the 90’s-00’s) we introduced the “TOGA doctrine” where data collected with NSF money had to be made public with all accompanying meta data within two years. That a condition of receiving NSF funds. I am not sure that is the case for derived fields but it should be.

David Palmer’s life would be dull without this ongoing dispute. I received the following preliminary answer to my request (which has been submitted under EIR). Readers should not react reflexively to the opening paragraph which merely allocates the response to the somewhat parallel EIR regime, rather than the FOI regime. The allocation between regimes is sometimes a bit hard to figure out, but EIR is fine with me.

I acknowledge your request for information received today, 26 June 2009. Your request for information has been initially considered under the Freedom of Information Act 2000 and we have determined that we are not obliged to supply the information you have requested under that Act.
The exemption applied is s. 39, exempting information that is ‘environmental information’ within the meaning of the Environmental Information Regulations 2004 from disclosure under the Freedom of Information Act 2000.

This exemption applies because ‘environmental information’ must be disclosed under the Environmental Information Regulations 2004 (EIR). In short, we will consider your request and provide information under EIR, not FOIA.

Your request is being considered under EIR and you will receive the information requested within the statutory timescale of 20 working days as defined by the EIR 2004, subject to the information not being exempt.
If appropriate, the information may be provided in paper copy, normal font size. If you require alternative formats, e.g. language, audio, large print, etc. then please let me know.

For your information, the EIR defines a number of exemptions which may prevent release of the information you have requested. There will be an assessment and if any of the exemption categories apply then the information will not be released. You will be informed if this is the case, including your rights of appeal.

If the information you request contains reference to a third party then they may be consulted prior to a decision being taken on whether or not to release the information to you. You will be informed if this is the case.

There may a fee payable for this information. This will be considered and you will be informed if a fee is payable. In this event the fee must be paid before the information is processed and released. The 20 working day time limit for responses is suspended until receipt of the payment.

I don’t have a magic decoder ring (or at least I don’t think I do). I just asked for some specific data and received it. I am a firm believer of open data. there can be no other way. years ago, we had scientists going out in the field writing papers and holding onto their data. As part of the TOGA experiment (big El Nino thingo in the 90’s-00’s) we introduced the “TOGA doctrine” where data collected with NSF money had to be made public with all accompanying meta data within two years. That a condition of receiving NSF funds. I am not sure that is the case for derived fields but it should be.

Peter

.
Much of the data in the climate science field is collected using NSF money … but that certainly hasn’t stopped researchers from hiding their data. What is of concern to me is that climate scientists are extremely unwilling to publicly decry the actions of those scientists. Phil Jones, for example, gives his data to people he approves of, and withholds it from others. I know he is not NSF funded, but the issue is the scientific method, not the funding source.
.
Your research, as I pointed out before, can never rise to the level of science. It will be allegory and anecdote until Phil publishes his data, because until then there is no way to replicate the work that you are doing. You are building your edifice on a pile of shifting, unpublished gray datasets with no backup, datasets that have never been publicly examined or verified. I don’t understand why the possible destruction of your work from being built on a foundation of sand is of so little concern to you. Perhaps you think that the current accolades are worth your findings being tossed in the trash later when the underlying data is shown to be incorrect or useless, I don’t know. But why take the chance?
.
In addition, although there are laws and regulations to prevent the hiding of data, the only way to stop it will be when climate scientists grow a pair and start publicly decrying the practice. Until climate science cleans up its own backyard by getting rid of the Manns and the Thompsons and the Joneses that hide their data with no complaints from the “scientists” in the field, the reputation of everyone in the field will be stained. Including your reputation. You are using data over which there is great dispute, data which is being refused to other people who actually care about science … and you stay schtumm, you don’t say a word, you don’t indicate in thought or deed that you see the slightest difficulty with Jones doing this.
.
Thats the problem that I see. It’s not that Mann and Thompson and Jones are trying to game the system, that happens in all human endeavors. The problem is that the “scientists” in the field are not doing what real scientists do. Thats why I put “scientists” in quotes, because you are not doing what scientists do. You are not cleaning up your own backyard. You are not calling the miscreants to task. You are not demanding that the data be archived and the secret methods made public. Quite the opposite, in fact. When the Barton Committee met to push Mann to reveal his dirty secrets, the community of climate “scientists” were up in arms about how it was wrong and unfair to the poor Mann … unfair to poor Mann? Get real. He’s a lowlife cheat and a liar, and none of the so-called “scientists” have the balls to say so. Turns my stomach to watch the gyrations that y’all go through to avoid actually saying that one of their own is doing something horribly wrong.
.
In any case, Peter, I respect you and your work. You have a good name in a bad field. Now, you say you are a “firm believer in open data” … ok, prove it. Put your money where your mouth is. If you are such a firm believer as you claim, write a letter to Jones saying “Hey, you gave data to me but not other scientists. What’s up with that?”. Write a letter to Thompson asking him to archive his datasets. Don’t just stand there holding your johnson and telling us you believe in open data, that’s useless. Show us you believe in open data by actually doing something. Become a real scientist and start helping us to clean up the mess in your own backyard. Because I don’t know about the others, but I’m getting very tired of doing your job for you and your fellow “scientists”.
.
The truth is that although you may be a “firm believer in open data”, until you actually do something about fixing the problems in your own chosen field you are and will remain, as they used to say on the ranch where I grew up, the kind of cowboy that is “All hat, and no cattle” …
.
w.

“We have asked Phil Jones for data so that we could compare the synthesized surface temperature with actual station data. Jones has provided everything that we have asked for. This is for our study of the 1930/40 climate bump that is ongoing. Alas, these things take time. But my experience has been quite different to yours. ”
.
An interesting juxtaposition: one of my reasons for doubting the accuracy of the corrected SST graphs is the way the climate bump is being smoothed away. Initially it looked as if it were solidly 1939 to ’45/46, but each correction (the Folland and Parker is particularly obvious) moves the beginning earlier and, by ramping up the earlier temperatures, makes the bump less of a feature. Even so, it still stands out from the temperature record like a canine’s apparatus and really should have been looked at before. It is not enough to simply dismiss it as ‘a series of super el Ninos’ as this is just explaining the fact it got warmer by saying ‘it got warmer’.

I had (lost on the old computer but the search continues — it was, IIRC, fao produced) a set of graphs to do with ocean productivity which showed windspeed variation neatly matching the original timescale, not the new improved 1935 onwards graphs and exhibiting an interesting distribution of highest in NH Atlantic and least in SH Pacific. Perhaps if the models produce windspeed variation outputs then their match to historical records will provide a verification of the new, adjusted timetable. Or not.

A matching exercise of 1935 to 1950 wind and temperature records from lighthouses to HADCRUT would also be instructive.

Re: Gary Strand (#123), clearly you haven’t been following the story. This is evidence by your question. You say:
.

Has anyone ever asked Phil Jones *why* he won’t release data to every Tom, Dick, and Harriet who asks for it? Instead of imagining his motives, why not simply ask?

.
Yes. Warwick Hughes asked. Jones famously replied:
.

Why should I make the data available to you, when your aim is to try and find something wrong with it.

.
Now, obviously neither you nor Jones understand how science works. So for your (and hopefully Phil’s) edification, here’s the short version of the scientific process:
.
1. Somebody makes a scientific claim based on some data.
.
2. Other scientists, both those who believe the claim and those that don’t believe the claim, try to find something wrong with the first scientist’s claims.
.
3. If they find something wrong, the claim is brought into question. If nobody can find anything wrong with the claim, it is provisionally accepted as fact. However, if somebody later finds something wrong with the claim, it is brought into question.
.
So Gary, perhaps you could show us that you understand this grade-school stuff by explaining to the class:
.
1. Whether Jone’s comment fits in any way into the scientific process.
.
2. Whether Jone’s temperature record has been verified by the scientific process.
.
3. Whether scientific information whose collection is paid for by your and my taxes should be kept secret.
.
And for your bonus question …
.
4. Should a scientist give his data only to scientists who are not trying to find something wrong with it?
.
Answer the questions to show us you’ve actually learned something, and people might actually start to pay attention to you …
.
w.
.
PS – If Gary refuses to answer the questions, I strongly recommend that any further posts on his part be roundly ignored. If he doesn’t answer and tries to post again, just reply “You can play after you’ve answered your questions, dear” …

Re: Gary Strand (#125), Per #100 Others have apparently been able to get requests fulfilled…the issue is really, why is he withholding from anyone at all? Being selective about it only makes it worse.

But you seem to think apriori presumably without knowing the circumstances, that Hughes had an agenda and that Jones was aware of it and not merely paranoid…you might be right (yes, he did have an agenda-who the hell doesn’t? And Jones didn’t necessarily know that…) but jumping to conclusions is inappropriate on such weak grounds. But all of that is irrelevant to the point addressed in the first half of this post…Quite simply, why is Jones so secretive towards some, not towards others, and why not totally open? I want to know what his motivations are, too…but I suspect that the quote already reveals it-Paranoia.

I think we should all be able to agree that his behavior is rather juvenile..regardless of whether you believe the HadCrut data are right or not.

Are you kidding? The agenda is to audit the data. If Phil Jones has a problem with that, then he has a problem with science. Openness to other researchers is one of the standards of science. To refuse access to data is THE mark of pseudoscience. Jones’ actions bring disrepute on scientists everywhere.

Given the timing of events, it is unlikely that Jones would have known if Hughes had an agenda or not. If he had noted some rather polite and normal questions from Hughes beforehand, he could hardly have deduced a “black” agenda from them, for this was before the years of much debate. AFAIK and that is reasonably well personally, Hughes has no preference for whether the temperature rises or falls. He’s simply interested in getting the science right. As are most who post here.

Gary, please don’t just sit there posing hypotheticals. If you have actual or original data analysis to add, then that is what is welcomed on this blog.

There’s a long history here which you’re probably unaware of. Steve and Willis (and others) have tried in vain to get Phil Jones’ data for years now. There have been no end of excuses and evasions, excemptions claimed under various regulations. When they wrote Hadley asking for that data, they were told it was provided to Hadley only if they refused to give it to anyone else. He picks and chooses who can have access to his data, denying it to those who might “look for something wrong with it”. I’m not sure what that is, but it isn’t science, and I can’t fault Willis for being exasperated by this years-long battle to get this data-set. Phil Jones’ behavior in this should be an embarassment to climate scientists of all stripes.

Re: Gary Strand (#129), you have missed the point. Whether Warwick has an agenda is absolutely immaterial. Science does not depend on whether or not you like another scientist. It depends on whether you have the balls to toss your ideas and methods and data out into the scientific marketplace for anyone to examine. Your friends. Your enemies. People with an agenda. Everyone. Public examination, not examination by people you have selected as Phil has done, is the essence of science.

As for condescending to you, I will continue to do so until you have the balls to answer questions and you can answer them in a way that shows you understand science. Science is a blood sport, not an old boys club where you get to pick and choose who gets to see your publicly funded data. Science means you believe enough in what you have done to be willing to expose your work to the harsh, pitiless glare of the world. Not just share it with your special friends. In fact the best people in the world to show it to are your enemies and people with an agenda. If they can’t poke holes in it, you’re home free …

But until then, you can go out and play after you have answered your questions, dear.

How many of you know *why* Phil Jones hasn’t released the data to any and all? Leave aside the acrimony and unwarranted extrapolations and personal feelings. Have any of you considered other reasons?

I don’t know why Jones doesn’t release the data – and lacking facts as to his reasons, I see no need to invent motives or ascribe personality flaws to the man. So many of you are vested in demonizing him that you’re no longer acting scientifically.

So, Willis, the lectures about being scientific are a waste of your time. Let’s just say your delivery needs some work.

Re: Gary Strand (#134), That is a ridiculous argument. Who cares what reason he has for not releasing it? In additional to the withholding of the data being anti-science (there can be no scientific reason for not releasing it), the data is being used to decide where trillions of dollars of public money is to be allocated.
.
Jones has a scientific obligation to release it. The US government has a legal obligation to ensure it is released and audited prior to using it for regulation. I don’t know the UK regulations well enough to make a statement there.
.
Regardless, your line of reasoning is to give a scientist the benefit of the doubt when he refuses to release information?
.
That’s absurd.

Re: Gary Strand (#134), I also see no need to invent motives or ascribe personality flaws to Phil Jones. However, I also see absolutely no reason to give his work any scientific relevance whatsoever until both data and method are clearly available. Until he starts playing science and stops playing politics, his work has no place in a scientific debate.
.
If he thinks his work relevant in the scientific arena, perhaps he should place it there.

You completely miss the point. There is no acceptable reason why a scientist would refuse to release data. Good reasons exists why a pseudoscientist would not release data, but none for a scientist.

If someone says to a scientist “I do not believe you. Some me your data!” A scientist would say “Of course. Here it is. Take a look for yourself. I have nothing to hide.”

A pseudoscientist would say “The data is not yours. Your request is improper. The data is too complex for you to understand. Pay no attention to the man behind the curtain. I am the great and powerful Oz who is here to tell you what to think!”

Let me rephrase. There is no acceptable reason for a climate scientist to refuse to release data. Scientists working on patentable inventions are required to work in secrecy. But if a scientist decides to publish his results, he has to release his data or he can expect to be pilloried.

Gary Strand, as an intellectual exercise, could you please construct a satisfactory reason *why* Phil Jones would not release the data to any and all? Perhaps there are even two or three very good reasons why this data that may be used to reshape the earth’s political systems might be reasonably withheld from those who will be affected. I’m looking forward to your comment.
Mike Bryant

RE 133 willis. thank you for trying to edify folks. Gary, one of the epistemological concerns with all science is observer bias. How is this addressed? It’s addressed by making your data and methods available to others. Because we cannot know motives, or observe motives, we can only control for motives. We control for motives ( observer bias for example) by allowing others access to our data and methods. When others, presumable with motives different than ours, repeat our experiments with the same results, then our results are deemed to have an increased probability of being true. In short, the burden of proof is upon Phil Jones. It’s his burden to prove that his results are independent of motive. he can do that by sharing his data and method. By failing to share his data and methods he removes his findings from consideration by other scientists. Phil Jones results have all the epistemological standing of an anecdote.

In the NAS Panel presentations in MArch 2006, Hans von Storch said in his presentation that he had read this comment and initially disbelieved that any scientist could have said such a thing; accordingly he confirmed with Jones that the comment was correct(Jones confirmed it) and expressed his extreme disapproval of this attitude to the NAS panel.

Re: Gary Strand (#145), No need to, just ask the question of your self: “Could there be any possible reason which would justify him doing so? Whether such a reason is actually his is beside the point. The particular reason doesn’t even matter. There just isn’t any reason at all that would justify withholding the data. The specific “rationale” is irrelevant. Like a crime, it is not the motive that matters but the actually violation of the law.

Imagine that as a defense attorney you tried to defend your client by saying you don’t think it is appropriate to speculate about why your client killed the victim, but then bizarrely claimed that the lack of a motive in the prosecution’s case invalidated their far more than circumstantial evidence of his guilt and proved his innocence.

Imagine that as a defense attorney you tried to defend your client by saying you don’t think it is appropriate to speculate about why your client killed the victim, but then bizarrely claimed that the lack of a motive in the prosecution’s case invalidated their far more than circumstantial evidence of his guilt and proved his innocence.

This is like the classic example of how not to argue in the alternative: the defence attorney, perhaps studying Team publications on the side, who argued that his client was not guilty, but, in the alternative, it was self-defence.

Cognitive dissonance would be an apt description of Gary’s position, if I am not mistaken, right Willis? :)
.

I suppose, however, one could say that Gary’s statement in #145 is actually a proper response: there is no need to “guess why Jones won’t release his data and/or methods” because Jones himself has already told us.
.

I have plenty of experience in the stock market. That’s how I got interested in climate science. Studies looked highly promotional to me – precisely because of that experience.

I’ve had the consistent premise that if climate scientists are dealing with the public, they ought to comply – at a minimum – with disclosure and due diligence standards that govern mining promoters. (Please do not bother bringing Bre-X or Lehmann Bros to my attention – I’m there way ahead of you.)

For some reason, members of the “Community” do not think that such minimum standards should apply to them – witness promotional press release after promotional press release, that would make a mining promoter blush,

Apparently CA is chock-full of mindreaders. You guys ought to be playing the stock market.

Your position consists of two fallacies.

First, it is straw man argument to claim that CA reader’s position consists of damning Jones’s motives for withholding data; in fact, CA readers are condemning the practice of withholding data, regardless of the motive for doing so.

Second, even if your straw man argument were true — that is, even if CA readers were improperly criticizing Jones’s motives — it would be pure non-sequitur to claim that such behavior by CA readers justifies Jones’ withholding of data.

In summary, the fallacious arguments you’ve advanced are transparent and lame attempts to rationalize the scientifically unforgivable practice of preventing others from testing one’s conclusions and assertions.

With the latest exposure of apparent corruption at the US EPA and the clear exaggeration of scientific conclusions in the EPA document combined with the flat statement that the Administration had decided to move forward on endangerment, don’t we at least get the chance to see the data from the REAL scientists? WTF, we’re supposed to accept ‘we told the truth’ as science. It’s data and DATA can’t do anything but support a scientists conclusion or prove them wrong—- right? Despite the obfuscation, there aren’t many reasons a scientist wouldn’t release the data.

Jeff, should the Carlin paper be part of the EPA peer review record? It’s not obvious to me that it should. There could easily be valid reasons why Carlin’s supervisor might decide not to include Carlin’s comments in his department’s position without there being any impropriety.

Having said that, I thought that Carlin’s supervisor’s correspondence was injudicious. While IMO he could have given valid reasons for not proceeding with Carlin’s remarks, his actual comments give the impression that he was bending EPA procedures to political will. Perhaps this happens all the time, but the proffered reasons remain somewhat injudicious.

Procedurally, it is my understanding that EPA is obliged to have a “peer review record”; OMB guidelines encourage the publication of the peer review record for a highly influential scientific assessment. We know that there were 12 “expert reviewers”.

I would urge you and others to FOI the peer review record from EPA.

Scientifically, there’s an almost total disconnect between the Carlin paper and issues that have been raised here over the past few years. These are definitely not the criticisms of IPCC that I would have made.

Scientifically, there’s an almost total disconnect between the Carlin paper and issues that have been raised here over the past few years. These are definitely not the criticisms of IPCC that I would have made.

They’re actually the ones that Pat Michaels made, almost verbatim, for example:

Steve, you suggest that Carlin’s supervisor’s remarks were “injudicious” by implying that
he was bending EPA procedures in response to political will. But what if that is in fact the case? Are you suggesting he should have given a politically correct response?

It seems to me that Carlin’s position is surprisingly parallel to your own, namely that the EPA should not simply accept the findings of the IPCC in it’s endangerment finding, but should be making its own evaluation, including later information and approriate peer review. I had read your submittal before Carlin’s document was published, and I thought that his document supported your case for proper peer review before reaching an important endangerment finding. That some of the scientific issues he raises are different than those you would choose doesn’t alter that fact in my opinion.