To meet 21st Century requirements it is necessary to reconsider our analyses of historical land surface temperature changes. This is about much more than simply re-engineering existing datasets. These datasets were adequate for assessing whether climate was changing at the global scale. This current exercise should not be interpreted as a fundamental questioning of these previous efforts. But these pre-existing datasets cannot answer all the questions that society is now quite rightly asking. They do not constitute a sufficiently large sample to truly understand our uncertainty at regional scales. At monthly resolution they are also of limited utility in characterising extremes in climate and their changes.

Regardless of the causes, climate is not and never has been stable. Changes in climate impact all of society. But it is not changes in the monthly mean at scales of hundreds of Km that impact us all. It is changes at much more local scales and that last a few hours or days that have a major impact upon society. Whilst the long-term changes characterised by the current datasets can ameliorate or exacerbate the effects these datasets are fundamentally ill-suited to meet most of our needs as a global society.

The challenge set out in the proposal from the Met Office to the Commission for Climatology (CCl, see background link) is to produce a new suite of datasets capable of answering these questions. This requires more than the work of a single institution to do at all. It certainly requires very many partners to do properly. Following the positive outcome of the CCl deliberations an organising committee has been convened and the Met Office will be hosting a workshop in September in Exeter, UK. The organising committee, with substantial international representation, includes a broad range of expertise and perspectives and is undertaking planning activities. As yet planning is at too early a stage to publish details. More specific information will be posted by mid-June. However, the expected outcome of this meeting will be an in-depth plan with multi-institution sign on as to how to proceed. Broad aspects to be covered will be:

The plan is to solicit white papers on the range of topics to be discussed in advance of the meeting and post these on a (moderated) blog for broad input so that non-participants in the meeting can still have some meaningful input. The invitee list includes representatives from a number of relevant disciplines including a number that need to be engaged if the project is to be a success: climatologists; metrologists (measurement scientists); and statisticians amongst others. To be effective the meeting will have to be relatively small but, as stated above, stringent efforts will be made to entrain input from non-attendees in advance. And, of course, participation in the work will not be limited to attendees of this initial planning meeting only – to be successful it needs lots of participants, many more than will be at the meeting.

Update: 8/20 New version of agenda and comment period will remain open until September 1st.

==============================================

Here’s my take on it:

1) The effort, while noble, is a reaction to a series of data transparency blunders rather than a proactive approach to open replication. In the original Fox News article I cited on Feb 23rd, 2010 they write:

At a meeting on Monday of about 150 climate scientists, representatives of Britain’s weather office quietly proposed that the world’s climatologists start all over again to produce a new trove of global temperature data that is open to public scrutiny and “rigorous” peer review.

After the firestorm of criticism called Climate-gate, the British government’s official Meteorological Office apparently has decided to wave a white flag and surrender.

While this effort is a step forward, it is unfortunate that it took Climategate to break free the idea of open and transparent data, and of surface data that has gone through rigorous quality control procedures. As we’ve seen recently, Canada’s own surface weather data is in such a mess that Environment Canada squelched their own embarrassing internal report and it took a freedom of information request to pry it loose. They called the state of the network “disturbing’.

A parallel effort as an integral part of establishing the databank is required to create an adjunct metadata databank that as comprehensively as feasible describes known changes in instrumentation, observing practices and siting at each site over time. This may include photographic evidence, digital images and archive materials but the essential elements should be in machine-readable form.

Is essentially a stamp of approval of my surfacestations.org project. Without knowing the changes in measurement conditions surrounding the century long experiment in climate monitoring, it is impossible to know the true quality of the data. I see this as a positive step forward.

3) Making this effort known to the climate community has apparently not been a strong suite of the Met Office, for example, I only found out about it a couple of days ago via a reporter asking questions about my views on it.

The Met office needs to be far more proactive in communications.

I encourage readers to make submissions before the Sept 1 deadline, as only a few days remain.

If the agenda behind this is to genuinely and honestly establish an accurate and truthful record of what global and regional temperature trends actually are, and they take a truly neutral and openly scientific approach according to the scientific method, then this can only be a good thing.

However, if they are starting with a hidden agenda to promote the “world is warming at a catastrophic rate” meme and they automatically reject any data that contradicts that view, then this will be a disaster.

If this is the former and the result is reliable and trustworthy data that shows categorically that the earth is warming at an alarming rate, then I shall accept it.

But, crucially, HOW are we more sceptical “laypersons” to know one way or the other if this is a genuine, honest, scientific search for truth instead of a slight of hand manipulation to endorse a political position?

Its easy to say open honest. The proof of the pudding will be in the composition of the working party, the transparency of their approach to dealing with historic temperature records, The handling and preservation of existing records and explanation/reasons and justifications for any proposed changes to be submitted for open discussion.

I would hope that the calibre of that proposed blog site and its moderation, will be leaps and bounds above the partisan effort of Real Climate and similar defenders of todays warming faith, if it is to achieve and re-instate credibility needed for the basics of weather science.

To be credible, they should have at least two other parties involved in transcribing the existing historical data into electronic form, to ensure there’s no “value adding”, “homogenizing” or any other mucking about.

Simply enter the date, time and temperature etc from the logbooks as written.

…Canada’s own surface weather data is in such a mess that Environment Canada squelched their own emabarrasing internal report and it took

…what?

REPLY: Cut and paste error. Here’s the corrected text.

As we’ve seen recently, Canada’s own surface weather data is in such a mess that Environment Canada squelched their own embarrassing internal report and it took a freedom of information request to pry it loose. They called the state of the network “disturbing’.

“At monthly resolution they are also of limited utility in characterising extremes in climate and their changes.”

“But it is not changes in the monthly mean at scales of hundreds of Km that impact us all. It is changes at much more local scales and that last a few hours or days that have a major impact upon society… ….are ill-suited to meet most of our needs as a global society.”

This is really easy folks: People are realizing that there is no reason to panic over the current rate of climate change and so there needs to be a terrifying dataset about some farmers dry fields and cottages hit by storm-fallen trees.

Apparently it is no longer the climate ppl have to worry about but the weather again. This has the added bonus that being a new kind of data, we will see “worst ever” scenarios happen pretty much all the time until the data accumulates in decades…

There are still two huge things missing that I want to see before I can believe them

(1) Sorry.

(2) Thank you Anthony for your Surface Stations work that brought all these problems to our notice.

It is shameful that professionals should need to be brought to heel by amateurs. And what I don’t understand is the reticence in apologizing. The disciplines of Counselling and Life Coaching both show that saying Sorry (and Thank You) are of health and wealth benefit, and most of all to the person who says Sorry.

Julia Slingo is still a warmist. Probably that goes together with the still-unsaid Sorry.

They have come down in the science scale since the 1966 WMO London conference on climate change 8,000-0 B.C. (where the first systematic data on the secular cycle that comprises such events like MWP and LIA was first recognized on a Global scale by B. Frenzel). Now they are known here only for poor summer-winter weather predictions, and are perhaps up for sale. Realistically though, one cannot expect such established institutions as the Met Office to make PUBLIC admissions or corrections of method errors with anything more than half-steps. It is not possible yet, in the dysfunctional climate ‘science’ climate for such institutions to admit gross error, so really, these half-steps to me show good faith and recognition of at least some error. Biped Hominids as a whole are clearly a group of highly socially IMMITATIVE species, and this group includes scientific workers of course. As this will be a multi-year project, in the PRIVATE minds of the project participants, there is possibly some realization that a sea-change of the research climate will take place this period, and that ‘alternate’ interpretation of the database can then take place without the kind of peer-harassment that would have emerged as recently as last year on this issue. I think that sea-change is already coming, and will grow stronger as the gov.-financial support of this CO2-based AGW nonsense slowly unravels.

I agree with Ken Hall. Unless they are prepared to start with a totally clean sheet — no preconceptions — then we are going to be back on the same old treadmill. It’s important that they understand why: if there is any downside to climate change of the magnitude that the IPCC and its adherents have been touting (and from what I have read here and elsewhere I don’t think there is) we need to be as certain as we can be that we are getting good data and good conclusions.
I’m interested in the following paragraph:Regardless of the causes, climate is not and never has been stable. Changes in climate impact all of society. But it is not changes in the monthly mean at scales of hundreds of Km that impact us all. It is changes at much more local scales and that last a few hours or days that have a major impact upon society.
This is the first published admission I have seen from the Met Office that climate “never has been” stable. But how are we supposed to interpret the last sentence? That’s not climate they’re talking about; that’s weather, by anybody’s definition surely?!
This has the makings of a Damascene conversion but the skeptic in me tells me not to hold my breath!

I agree with Lucy – replace the hubris with a little humility as a first step to regaining trust. The politically correct terminology of the press release seems to be couched so as to not ruffle anyone’s feathers. However, to state that the current datasets have been adequate on a global scale is clearly a crock. Again, I agree with Lucy.

Wikipedia: “Hubris often indicates being out of touch with reality and overestimating one’s own competence or capabilities, especially for people in positions of power.”

It seems that this exercise will enable the Met. Office to reduce the Roman and Medieval Warm Periods temperatures and rates of temperature rise to prove that the 20th century is warmer and temperature rise faster than any other due to AGW. They will out Gore Gore.

Here in Britain the Met Office is a bit of a joke. Their long term forecasts since about 1999 have been increasingly bizarre. They predicted several “Barbecue Summers” in the spring only for the entire summer to be a washout. They told us that snow is a thing of the past – with the result that local government stopped stockpiling salt and grit for the roads – just before the entire country was paralysed by a massive fall of the stuff. Then they scrapped their long term forecasts without explanation and without saying “Sorry” for all the trouble they had caused. Of course, their predictions of hot summers and mild winters got loads of media coverage about global warming, while the reality was barely mentioned.

Just for your info, our summer holiday in Wales was a washout. Only one day of sunny weather to take the children to the beach.

It seems surprising to me that this story wasn’t run prominently in all the major UK newspaper titles a few months ago. I admit, it’s possible that it was and I missed it, but I’m sure you lot wouldn’t have.

I’m all for a global data set, but if there is only to be one, then it must be subject to the most rigorous transparency and scientific critique possible, because if it isn’t, it’ll become like a religion. And climate data is too important to become a religion with a modern day set of Knights Templar keeping their secret rituals under lock and key….

In bureaucrat language this is an admission of failure and a new start – it seems positive to me.
Also the (current) UK gov has some smart, skeptical people briefing on this issue (Lord Lawson, etc.), and are essentially realists IMO. They are somewhat beholden to the Green movement as they’re dependant on the liberals for the majority.
I expect and hope that the absurd 80% CO2 reduction policy will quietly be shelved, and some of the career alarmists at the Met will be “involved” in the 20% reduction planned in state workers.

Why does the presence of the word “homogenisation” make the hairs on the back of my neck stand up?

And exactly how are they going to get such minute data over time and space for the whole planet from 1700? 1800? 1900?

Climate is NOT the 30 year average of weather. The 30 year average of weather is still weather. (There are longer weather cycles such as the PDO, AMO, etc.) Climate is a 30,000 year scale event. Until they get past this fundamental breakage of their words, they will never get it right. All they will find is long duration weather cycles that scare themselves.

The Sahara has been the Sahara for a few thousand years, and is still now.
The Mediterranean has been a “Mediterranean Climate Zone” for thousands of years, during the MWP and the LIA. And is still now.
Russian taiga has been taiga as long as there have been Russians, and is now.

So one hopes they realize they are finding that weather changes, and scaring themselves about it. Looking at it with even higher magnification will just make the bugs look bigger. They need to step back and see the big picture.

“But these pre-existing datasets cannot answer all the questions that society is now quite rightly asking.”
As in why do they keep telling me that this is the warmest year since forever, when I’m freezing my bollocks off? They have realized that the unwashed don’t really understand the difference between weather and climate, relying on memory from a purely local perspective, which is always susceptible to personal bias. A local dataset will provide a double edged sword in as much as local cooling can be explained as merely “local” and just weather. Alternatively, a local warming in the dataset will definitely be proof of broader climate change and an invaluable tool in the propaganda campaign. That said I believe that an attempt to “clean up” the record is desirable as long as the “cleaning” is transparent and open to all stakeholders. Lastly, however as in most human endeavors where there exists an agenda or the potential to influence “he who keeps the secrets, controls the answers”.

I apologise for being Mr Picky yet again! Why do they get some half-wit to write this verbose crap? It’s full of wish-washy overly wordy phrases that it takes most people, well me anyhow, ages to try & work out what they are actually saying in plain English (an unwelcome return of Sir Humphry Appleby?). Surely the starting of two sentences with the word “But” is appalling English, followed by “from a number of relevant disciplines including a number that need to be engaged…..” Could they not have used the words, ” some”, “several”, “many”, or ” a few”, or “those”? I know my niece could have done a far superior job of written English than the numpty/muppet who wrote this stuff when she was about 12 years old! My English teacher would be turning in his grave. When I was emlpoyed at Rutherford Appleton Laboratory in the Council Works Unit, (many moons ago) we had a limited budget for building & engineering works. However, every letter/paper/report was checked & double-checked for grammar & spelling so that nothing ever left the building without being correct on that score. It was a matter of pride, you see. Today it seems, having a $235M budget & 1800 employees, & a £30M X-Box360 means don’t bother about the English! Huh!

Do any of the current collections of temp records give both the raw data and the code they used to adjust it, in a reproducible format? Or do they all include unexplained manual adjustments?

I think one of the important parts of these temp records is that scans of the original paper documents, where used, should be attached to the data files. Checking every temp in every paper document might be impractical, but we could check a random sample to evaluate accuracy of transcription and original sourcing. It is also important that all data from stations that are not used also be included to see if selection is biasing the results.

Ah the vaunted Met Office who if I recall correctly forecast a BBQ summer for 2010. Well, credit where credit is due, May and June were pretty good, but over the latter half of July and into August it has been less than brilliant. Not their fault of course, the weather does its own thing, but what it does suggest is that our highly paid folk in the Met Office have much work to do to regain even a semblance of credibility. Perhaps this is a start, one would hope so, though they will have discard that willingness to act as political poodles, not easy in intensely politicised Britain.

@Christopher: The MET office climate change graph you link to is accompanied by:

“The stations used to compile CET are chosen from the UK surface station network to be consistent as possible with those used historically. The data are then adjusted to ensure consistency with the historical series.”

As far as I am aware, this Met Office website is breaking UK law. Since 2007 all UK-based websites must provide:
■Name of the company providing the service (any difference from the trading name must be explained)
■Geographic Address – Not just a P.O. Box number. If the business is a company then include the registered office.
■Email Address

Just for your info, our summer holiday in Wales was a washout. Only one day of sunny weather to take the children to the beach.
(sorry, can’t do italics on iPad)

Huth says: Standard. Scotland likewise. West coast UK, bang in the middle of the Gulf Stream. Can’t ‘expect’ anything else, whether you call it weather or climate. Rain is only a problem here if you don’t like rain. Guess why Britian and Ireland are so green all summer.

something no one seems to be asking.
ok so they are scrapping? ALL the prior data.
now, is’nt that such a handy way to be able to NOT ever provide the real figures from the last X Years they supposedly used to “prove” the Gorewarm period.
It enables the offenders and the way they did it to remain ever hidden.
Get out of Jail ,Free card indeed.
the admission that they are complete duds at anything mre than a local weather for the day reporting..
Why? do they suddenly need such sharing with overseas to track and note and colate some TRUE temps for such a tiny country?
when for some reason Aus and NZ were previously sending OUR temps there to be screwed round with .
To me it smacks of a real “cold collation” of AGW specialists,
who will be planning to use their so called international collaboration of many and varied etc etc to make a concerted effort to silence any dissent now and forever after…
and the result, while we freeze our bu**s off, will be its still getting hotter meltier whatever suits their Carbon Cash whimsy!
the Moderated comments. yeah well apart from bad language I wonder will “bad questions?” also be removed, like what ARE the exact temps in my country, BEFORE any tweaking? for EVERY station. and what WERE they for the last century.
simple stuff really.

Why are we wasting our tax money in doing this at all?
So they want a mean value for the world temperature, and then they can say, “Oh the temperature was up 0.01 degreeC this year” – So bloody what!

In 1947, The River Thames flooded really badly, it cost a fortune in damaged property, and was caused by a long winter with heavy snowfall, suddenly melting after the arrival of a warm storm from from the Atlantic. The runoff was catastrophic.
For the last 60 years the Thames has had few floods, and none as costly as 1947, and that is because many schemes have been in place all along the river, for flood alieviation and prevention, in some areas, simple weirs have been constructed, and in others open canals with sluices.
My point is, that this is real environmental work which protects peoples lives and property, and is how we should be spending our taxes, not fiddling about with homogenising data measure in 1860, in the forlorn hope that it will confess.

I am sure that if you ask them, the current citizens of Pakistan, will assure you that they have no desire or need to know the current mean temperature of the planet, but they might have an interest in doing something on the lines of the Thames above.

We need to get our priorities right, and stop chasing a will o’ the whisp.

They do not constitute a sufficiently large sample to truly understand our uncertainty at regional scales. At monthly resolution they are also of limited utility in characterising extremes in climate and their changes.

Surely this quote means that they are going to “find” some data to help draw better conclusions about what we know we saw. Well surely we will have seen it when they are finished…!

Is the Ministry of Truth driving this improvement? There is an awful lot of History to “fix”. Do we have the money to rewrite all those reports and history books? Did Lord Monckton really call for the records to show an additional two degrees of additional warming so as to correctly state the 2009 values? Did Stephen McIntryre really validate the Hockey Stick? Well only Future History will demonstrate these new truths…

The problem :
But it is not changes in the monthly mean at scales of hundreds of Km that impact us all. It is changes at much more local scales and that last a few hours or days that have a major impact upon society.

“There are still two huge things missing that I want to see before I can believe them

(1) Sorry.

(2) Thank you Anthony for your Surface Stations work that brought all these problems to our notice.

It is shameful that professionals should need to be brought to heel by amateurs. And what I don’t understand is the reticence in apologizing. The disciplines of Counselling and Life Coaching both show that saying Sorry (and Thank You) are of health and wealth benefit, and most of all to the person who says Sorry.”

It’s obvious what is behind this. The world recent temperature trend is about to be exposed as steady or declining, and they know they can’t stop this. So they are starting a new tactic, that is, the frequency of extreme weather is the new climate change.

Last November or December they forecast that 2010 would be the warmest year ever. I am not aware that they have even backtracked on that yet, let alone apologised for it. Is this the organisation to be trusted to set up a new system?

There can be no doubt that a new system is needed but if it is to be worth anything it must have planned continuity at its heart (to say nothing of meeting ordinary statistical sampling criteria). By the time the record is long enough to be greatly useful my current great grandchildren will have grandchildren of their own.

What, as a first step: Reading, DegF/DegG, Date&Time, Coordinates, Instrument type etc. Where the reading is the one taken by the bloke/actual instrument output? All assembled into a searchable database & published online?

“But it is not changes in the monthly mean at scales of hundreds of Km that impact us all. It is changes at much more local scales and that last a few hours or days that have a major impact upon society”

Re the HadCET figures, as per Christopher’s link..
The ‘Central England’ area identified on the map is almost one giant suburb now – do we know whether any (or most) of the measuring stations are now in ‘warmer’ spots..?

Not only did the Uk local authorities fail to be wise virgins and stock up on grit before the “never-again” snowfalls here, thanks to the Met Office’s uselessness but it turns out that the Met office is also clearly responsible for many unwise and non-virginal citizens’ abject failure to supply themselves with adequate contraception:

The CET area now has many many millions more houses built on it in the last fifty years or so , each house being heated to sixty or seventy degrees for at least six months of every year . No matter which way the wind blows this warm plume of waste heat must have been be effecting thermometer readings , or is the increasing density of our populations and their extra winter heating requirements taken into account and the extra artificial heating discounted ???

I too thought the piece was filled with benevolent governmental puffed up phrasing, and cringingly overt present day jargon. I also thought that Brits willing to pay for this group to do anything other than pick up their pink slips is the best definition of insanity I have ever read. It left my brain reeling with pithy little ditties: Head, meet brick wall; fool me once shame on me; and all that rot.

What is the purpose of this project?. Strategic objectives etc. Is it to continue the lies and manipulations of the past but in a different guise or is it at last to produce a worldwide, high quality database of raw weather data for public and private analysis (specifically independent of world government interference) or is it yet another attempt to convince the idiotic, unknowledgeable public like me (BSc MSc Physics) of the reality of CO² anthro- global warming (not). Your reputation worldwide among the greater scientific community is totally in ruins. Your warmist exaggerations both recent and in the past have demonstrated a willingness to lie at all cost in order to promote your agenda of global warming (for it is global warming and not climate change that you promote). You have one more chance to put together a group of respected mathematicians and scientist of unbiased and unsullied reputation (not to be confused with the 3 recent enquiries into CRU-EAU which were none of those things). You have not started well with the inclusion of Stott and Thorne. Hopefully you will not continue in this vein for if you do all will be lost.

I wish the team all possible success in their endeavour to produce a clean, unmolested, unadjusted database. We do not want or need “value-added data”, please. We will all be watching and waiting with baited breath.

Tony Armstrong. ‘Fraid not! That doesn’t apply, as the site is a website within its own right – because it has a separate domain name. There is no ‘Contact us’ on the surfacetemperatures.org site! It’s breaking UK law as it stands.

David, the CET record has been corrected for UHI (by certain Phil Jones btw ;)). Compared with truly rural Armagh Observatory in Ireland, there is no trend in their mutual difference so it is quite good representation of European/NH climate.

The “do-over” of the surface temperature record has been in the making for quite a few month. I wonder for how long the Met Office have been at work to craft the announcement of the conference at which the constraints and specifications for the “do-over” will be hammered out. Moreover, I am concerned about the short time interval remaining between the release of the announcement now and the start of the conference on the first of September. The extremely short interval cannot possibly have been an outcome of an accident or because the Met Office were pressed for time.

Every single sentence in the write-up was put together with great care to express exactly what it is intended to bring across. The trick in reading the write-up is to determine not so much what is being said but what is not being said. Taking just the sentences in the first paragraph:

“To meet 21st Century requirements it is necessary to reconsider our analyses of historical land surface temperature changes.”

That means not changing meteorological data records but reconsidering how they analyzed them before and whether the conclusions reached before will still be promoted in the future.

“This is about much more than simply re-engineering existing datasets.”

That means the opposite of the first sentence, namely that the data sets will not only be changed, they will be *re-engineered.*

“These datasets were adequate for assessing whether climate was changing at the global scale.”

Aha! There was truly nothing wrong with the data they used before. The data did prove that there was climate change, namely global warming.

“This current exercise should not be interpreted as a fundamental questioning of these previous efforts.”

Naturally not, after all, it would not do to admit that anything was wrong with what the Met Office previously did.

“But these pre-existing datasets cannot answer all the questions that society is now quite rightly asking.”

Some ‘value’ needs to be added, perhaps extensive ‘re-engineering of the data’ probably cannot be ruled out, but let there be no mistake. The Met Office will not necessarily address questions society should be asking, it will address the questions “society is now quite rightly asking.” It seems to me that there is a critical distinction to be made between what is logically required and what is necessary to satisfy and perhaps indoctrinate (as before) the unwashed masses.

“They do not constitute a sufficiently large sample to truly understand our uncertainty at regional scales.”

That means that they must 1.) select a much larger sample for analysis from the data records created by weather stations all over the world, or 2.) manufacture data out of thin air (as E. M. Smith pointed out, homogenizing is a red flag, and that is mentioned later in the write-up), or 3.) a combination of 1.), 2.) and perhaps something else.

“At monthly resolution they are also of limited utility in characterising extremes in climate and their changes.”

Beyond insinuating that the resolution will be better than monthly, does that mean that they will produce re-engineered data at weekly, daily or hourly resolution? For instance, will they show daily minima, maxima, averages and standard deviations plus moving averages of temperatures?

The announcement does not inspire confidence and raises a lot of questions. I hope that the people going to the conference will get a chance to ask all of the questions that concern them and be successful in having them all addressed. More importantly, I hope that none of the answers they expect to get will be ruled on account of a preconceived agenda that is being advanced by the forces that steer and control the Met Office. Those things happen. They are called “bargaining in bad faith.”

Lucy Skywalker says:
August 25, 2010 at 1:07 am
There are still two huge things missing that I want to see before I can believe them

(1) Sorry.

(2) Thank you Anthony for your Surface Stations work that brought all these problems to our notice.

It is shameful that professionals should need to be brought to heel by amateurs. And what I don’t understand is the reticence in apologizing.

=================================

I could not agree more, Lucy.

What is even doubly shameful is that these people are “public servants” that is, not only are they not saying “sorry”, but they are doing so with taxpayer, public money.

Public servants have NO right…NOT to say sorry. They are not their own. They are owned by the public.

What sheer utter, and unfounded arrogance.

Again…as I said before….no wonder they are up for the sale chopping block.

Maybe, once they are sold and they are in private hands and have to depend on PRIVATE money like in the real world, will they become a little more real-world scientific and darwinian, in their approach!

Maybe the US should sell NOAA and NASA to the highest American bidder.

Maybe Canada should sell Environment Canada to the highest Canadian bidder, and so on.

I agree with Galane (August 25, 2010 at 12:52 am)
“To be credible, they should have at least two other parties involved in transcribing the existing historical data into electronic form, to ensure there’s no “value adding”, “homogenizing” or any other mucking about.

Simply enter the date, time and temperature etc from the logbooks as written.”

Nothing the MET office can do today ALONE can be trusted. If they do not join with others who retain scientific credibility, their new (taxpayer/government supported?) efforts cannot be respected.

Where to begin? With the last Ice Age, which we are still recovering from? Or the fact that atmospheric temperature changes are 1000 times less relevant than oceanic temperatures? Ignoring of enthalpy and humidity in the temperature record? The systematic destruction of Science–correlation taken as causation, consensus elevated to god-hood, and ad hominem argument endorsed as logic? Or the suppression of dissent, and incestuous peer review? Or the whitewashes that have established governmental units as full participants in Climategate after the fact? The system is rotten to the core; why should we expect this part of it will be any different? Without full public, front-page MEDIA REPORTED apologies, the credibility of all these governmental, journalistic, and climatological bedfellows must remain exactly zero.

The met office has blown any credibility it had IMHO, too many ridiculous assertions based on a political narrative and too many false predictions based on flawed data and methods.
The met office led and directed by by possibly the worst leader ever placed in such an office and the reliance on a very expensive super computer running corrupted models has destroyed a magnificent national institution and sullied its reputation as the father and mother of the art of meteorology.
When I think of hundreds of dedicated staff stretching back to the dawn of the industrial age and how they would feel now if they could but regard the wreckage that is the met office of today, what a tragedy.
It will take years of effort and a detailed reformation to rebuild the met office to a shadow of its former self, if they ever do.

I just threw some back of the envelope figures together about the costs of a world temperature database. For background, I’ve been a database administrator, architect and analyst since 1998, working on enterprise databases for a very large firm, so I know a little bit about that of which I speak.

First I took an impossibly dense network of temperature reporting stations: starting with one exactly on the North Pole, and extending southward on concentric latitudinal great circles 5km apart. That is, one at the pole, then four along a circle 5km radius from the pole, then seven along a circle 10km radius from the pole, etc, down to the equator. Unless I dropped a decimal point somewhere or went way off the tracks somewhere else, I got roughly 50 million stations to cover the entire earth with reporting stations within 5km of each other — each station covering an area of about 78.5 square km.

If you built a database with a 200 byte record length for each station to record that station’s daily highs and lows — along with identifying information on each station — you’d have 50 million records/day X 200bytes/record = 10 gigabytes/day for storage. Mutiply that times 365 days a year and you have 3.65 TB of data/year of raw temperature records for the Earth.

A two terabyte hard drive online is priced at $110, so it would cost about $220/year to store the data.

There’s a lot of other database considerations to take into account — but remember, that’s using a completely unachievable level of data collection covering the entire surface of the Earth. What we actually can do would use a lot less data storage.

In the field, we use that hoary old expression “He knows just enough about databases to be dangerous.” I’d bet every field uses a variation on that theme. And that’s just what the state of world temperature data looks like to me. My heart went out to that poor database guy in the emails who bemoaned the state of the data he was trying to make sense of — I see that sort of mess all the time as well.

To end this tome, I’ll just add that what needs to be done is to let some actual database professionals design and implement the next generation of temperature and climate records. This data is too important to be left to the amateurs.

“This current exercise should not be interpreted as a fundamental questioning of
these previous efforts.”

Our “engineered” data has been shown to be flawed and inadequate, but the conclusions we drew from that data is not to be questioned.

As if, “we KNOW that the sun revolves around the earth, and those who question this are heretics. We simply are having a little trouble explaining the motions of the planets. We will re-engineer our measurements till we get it right.”

Thanks Juraj it’s useful to know that the Central England Temperatures as featured on the Met Office website are both adjusted to take account of the urban heat island affect and that the record is representative of Northern Hemisphere/Europe temperatures generally. But, in which case how does the CO2/AGW theory explain the past decade’s rather rapid and sustained cooling in the UK and across the NH?

I still don’t know why the Wet Office et al insists Solar effects are not responsible for any apparent warming. As I have said before, the illogical attitude of their arguements when the IPCC reports openly state in the tables of all forcings, that there is a very low (UNIPCC 2001) to low (UNIPCC 2007) level of scientific understanding. This of course does give them a back door out of the room when the elephant crashes in! However, (not “But”), it’s a case of petitio principii, if you have a low to very low level of scientific understanding about something or anything for that matter, why play the ostrich & display an utterly cavalier disregard for its potential effects on climate? (The Wet Office, not the ostrich!) From personal experience when talking with my Wet Office fellow chorister earlier this year, he told me that the Wet Office does have top astrophysicists looking at possible solar effects all the time. Well, they’re either rather slow learners, or aren’t too bright, or somebody doesn’t want them to find any potential causal link! The exact nature of the Sun’s effects on climate may never be known, but to deny that there is any effect is just stupid.

It will be interesting to see if Britain gets another chilly blast from here there is no ice, if a similar situation occurs re salt-grit stocks. Already this year people were injured & some killed in road accidents due to treacherous conditions & roads had not been gritted. The opporunity to sue will not be far away once the blame game starts in ernest!

we can hope that more historical data will be brought into the system. There never has been any “dropping” of data or stations. Just an inglorious task of historical research that stopped for GHCN in the 90s.

I think its better if we stop refering the change in number of stations as a “drop” as if they had the data in the database and dropped it.

As long as they are going to reconstruct the data, how about assigning a confidence level to each temperature data item this time? That way climate models using the data can report how much of their model result is reliant on interpolated, guessed, corrupted, or non-existant historical temperature measurements.

Aren’t there pristine, rural sites with long, continuous records throughout the world which haven’t been contaminated by UHI effects, and don’t the climate record centers have that data in their possession? Can’t we just see the raw data from these pristine stations, and then perhaps we will know how much temperatures have risen. It seems to me that most of the climate scientists working in the public domain, and receiving their funding and salaries from the public, should not have a problem revealing their raw data, as:

1. The public paid for it, and should have access to the data.
2. IMO, these folks are in violation of the law.
3. It is the ethical thing to do.
4. What are they trying to hide?

Of course, I know what the real problem is. Data from pristine stations are very likely to show only a slight warming, which is entirely consistent with what I and most others on this blog beleive. I have no doubt the world is warming slightly, as it has been pretty consistently since the LIA, and has been in a general long-term warming since the last glacial period.

But to turn the world upside down because of a slight warming which is consistent with what has happened in the past is for me not justified. But I do have an open mind to people who would engage me in a respectful, decent manner, exactly the way Dr. Curry did on two threads right here on WUWT last week. She made her case for her hypothesis in a forthright, professional manner. Are you listening, Phil?

I think Walter Schneider 6.59 25/08 is absolutely correct in his analysis of the press release. We in the UK have had a decade and more of this type of carefully crafted spin from our former NewLiebor masters and their diktocrats. This double speak jargon is horribly like the outcome of the reports into the Iraq war (Hutton/Butler) for instance or the recent climate gate whitewash.

Every time you read one of these statements you must ask “what are they NOT saying” rather than “what are they saying”. I only hope that Anthony and the other sceptic heavy hitters can influence events enough to at least make the data/process transparent as the Met Office have already decided upon their conclusions.

Interesting comment that more, as well as better, data is needed. What does this say about the GISS/NOAA data in general, and the massive dropout of data in the last few years? Hansen has said that the dropout is not important after a stastistical review. Presumably a properly selected single station could show the trend. The English Met Office seems to have a different view.

They are wrong from the getgo. There is no mention of revisiting the papers that established the adjustments including UHI and homogenisation. Without do overs of this most important basis for the temperature record they have no credibility and the data they produce will be contentious and unsupported except by warmers and fanatics.

In other words they simply stated they would create more corrupt data sets.

I applaud you vigorously, Anthony. Your surface stations project is turning out to be a truly great service to mankind. It has long been there, in their faces, begging much deserved attention. But now it is actually forcing accountability on this tightly controlled mass movement.

Their ultimate paradise:
History will tell, perhaps in less than a hundred years: First our forefathers started with the homogenization of temperatures, then, as time passed and they realized that there were inconvenient attitudes among some people then called “deniers”, they wisely decided to homogenize, also, the genus homo-sapiens into three well differentiated classes: The higher class, that of the leaders, called the most highly and venerable ALPHA CLASS, was formed by the direct descendants of our forefathers without any genetical manipulation at all. The lower second class, the class destined for the honorable service of the higher class, were the BETA CLASS, genetically managed so as to make them, at the same time, faithful servants and believers. The third class, that of slaves, called the GAMMA class, was engineered as not to think or discourse anything whatsoever, so they were born tongueless and genetically modified as to have strong and resistant bodies …..

To be successful this project needs buy-in from ALL interested parties. One crucial aspect is to assemble a group of such parties (listed with their credentials) – from professional societies to bloggers – and give them an opportunity to attach their own Seal-of-Approval (or quality rating) on individual datasets. This would be done after examination and testing using whatever documented criteria they like. There will be datasets everbody can agree on and datasets nobody puts much confidence in. Most will have a range of disagreement, but at least a preponderance of support or rejection. Evaluations might be more specific than a simple approval or rating. Categories of evaluation might include provenance of the data, metadata, proportion of missing or uncorrected data, etc.

Sure, it’s a consensus evaluation, but for basic data that’s mostly a good thing. Researchers then can decide which datasets to use and what criticisms will need to be addressed right from the beginning.

Lucy:
“It is shameful that professionals should need to be brought to heel by amateurs. ”

A strong element of truth, but also a trap. I am an amateur. Watts, Pielke, Wegener, Linzen, Spencer, Grey–It’s a very long list– are not. They bring the skill of their respective disciplines to the discussion. Don’t let the alarmist crowd dismiss dissenting views as the work of mere amateurs.
John

“There never has been any “dropping” of data or stations. Just an inglorious task of historical research that stopped for GHCN in the 90s.”

I think you need to take a remedial english course along with those remedial physics courses. They were using data from particular stations. They then were NOT using data from all the same stations. The stations, whose data they stopped using, were DROPPED!!! You need to say what you mean. There is no empirical evidence that the stations were PURPOSEFULLY DROPPED!! In fact, with the adjustments that are applied to stations, whether they are needed or not, it is likely that they may have reduced the trend by allowing the drop of stations. But then, if they had not allowed their trend to be reduced it would be pretty darned obvious that their series was flawed. What would be happening right now if the GISS and HadCrut series were 3c/C and the Satellites weren’t??

Of course this shows that creating a factual, accurate temperature series was not high on their priority list or they would have been paying more attention to the details and this large DROP of stations would not have been allowed! We see this with many of the primary data series from the Climate Community. They are apparently quite happy with the data until it conflicts with their models, at which time they suddenly find enought time and money to fix or “ADJUST” the data or reANALyse it!!

Now, you and your buddies can reconstruct the temp records all you want and all you are showing is that you are sycophants to the big boys. When you start addressing how the adjustments are done and how they are actually implemented on the raw data in a meaningful fashion you will be doing something USEFUL!!!

Remember, GIGO. Your temp series is just as useless as the Official ones for just this reason.

How come they don’t SPECIFICALLY state:
*With all data freely available/downloadable
*With all data transformation processes easily reproduced and open to the public
*With data collection processes and their inherent limitations
*With monitored public forums

Are we supposed to trust them with “communication, engagement and auditability?”

Those words can be true within a faulty peer-review process, including only hand-picked auditors.

I’m not impressed. They’re not as forthcoming about the openness I was expecting–and the scientific community should require.

I wish I had joined this debate earlier, but it has all been said above.

The Met Office has disgraced UK science, along with a “mickey mouse” university in East Anglia and the once noble Royal Society (Newton must be turning in his grave!). Out UK taxes have risen because of these idiots and 1600 decent people lost employment in Redcar at a 100 year old steel works in order that “carbon credits” could be cashed in by India.

I would urge every UK reader of this website to write to the Prime Minister about this travesty. I cerainly will (again!).

It is most certainly TRUE that GHCN does not carry forward all stations post-1990/91 that report in the CLIMATs. Station 51463 has no data in GHCN post-1991 but it has almost a complete record through 2010 in the CLIMATs, so it has been “dropped”.

The much vaunted C E T record is just a crock, over the years the recording stations have changed geographic location many times and those that remained in the same location for a period have been built around , found themselves in the middle of an international airport or downwind from the UK’s busiest motorway.
As an historic ‘continuous’ record it is meaningless.

Anthony, a couple of suggestions as you consider your input to surfacetemperatures.org.

Assuming that they do include adequate daily temperature data (yes, big if), I’d like to see them add meta-data for the annual (or decadal) population figures for each station, so that UHI effects can be adjusted for. (For airport sites, that might be average number of daily flights.)

And, if not already included, they should add the altitude of the station, in addition to the latitude, so that benchmarking and adjustment for this might be possible.

1st commandment of intelligent people: DO NOT BELIEVE IN NAMES, because it is a fallacy by definition.The modern science reader simply has too much trust in the big names. http://www.milesmathis.com/tide5.html
They want you to believe in them and in the names of their supposedly HOLY and UNMISTAKABLE institutions. Tell them you have the biggest and most humane name of all they should be, also, to believe in: LIBERTY.

Actually the experts they employ in this makeover should NOT include any of the usual suspects. Why would independent mathematicians, physicists, geologists, archeologists, historians…. not be trusted to do this work. What kind of knowledge is required – how to read a thermometer? How to guage whether the record is skewed by urbanization, how to deal with missing data. Also, I believe Finland or Sweden have a 200 year old plus temp record with a series on the ice break-up in the Baltic. The Irish have a comparatively long series. The US has a long series and there are long series in the Arctic that haven’t been compromised- particularly in eastern Greenland plus Arctic and Antarctic expeditions’ measurements (with the exception of the ignominious Catlin Expeditions).

I’m afraid I don’t trust them. They already have three “whitewashes” under their belts to embolden them to do another one. I believe skeptics and uncommitted should do a parallel study using all the long series data. Publicize this and serve notice that we will have a “referee” set of data to judge them by.

And he’s still hard at work waving his shrouds around. Check out:-http://www.zcb2030.org/downloads/ZCB2030.pdf is the 384 page report with a foreword by – Sir John Houghton and ‘funded and partnered’ by (amongst others)
The Met Office (and UEA!).

This has to be one of the most tendentious and dishonest “reports” ever written. An eco-fascist “Mein Kampf”. It pretends that the UK could not only be “zero carbon” by 2030 (without nuclear) and that this would be “affordable” and “create jobs”.

A review of world temps headed by the UK MO. Can anyone spell UK HMG funded (to the tune of many hundreds of millions of dollars) arm of the WWF. But let us assume for the moment that they want to come clean and even discover an error of two tenths of a degree (not likely but.. either way, irrelevant). So what? My only shock is that the Earth can maintain temps over hundreds of years to within a few tenths of a degree.

That aside, what exactly are they going to prove? Perhaps that temps have increased since the very well documented LIA or that it is warmer now than it was when Skara Brae was abandoned some 4000 years ago (but not when the settlement was established). Perhaps they can explain why it seems so crucial [in the climate science community] to remove many tenths from the raw record of the 1930/40’s.

When you get right down to the bone though, what it is all about is keeping us all arguing about FDM versus RSM and tenths of a degree here and there while WWF, Goldman Sachs, BP, Shell, Deutsche Bank…. all make you work an extra day a week for no return other than paying your “carbon obligation”. Rent seekers, Politicians and their “useful idiots”.

What these thieves should remember is that “taxation” without representation never ends well. The Guillotine rarely cuts based on what part you played – unlike “climate” it’s a kind of binary judgement.

Cassandra King says:
August 25, 2010 at 7:44 am
The met office has blown any credibility it had IMHO, too many ridiculous assertions based on a political narrative and too many false predictions based on flawed data and methods.
The met office led and directed by by possibly the worst leader ever placed in such an office and the reliance on a very expensive super computer running corrupted models has destroyed a magnificent national institution and sullied its reputation as the father and mother of the art of meteorology.
When I think of hundreds of dedicated staff stretching back to the dawn of the industrial age and how they would feel now if they could but regard the wreckage that is the met office of today, what a tragedy.
It will take years of effort and a detailed reformation to rebuild the met office to a shadow of its former self, if they ever do.

Did you know that the Met Office was started by Robert Fitzroy, Charles Darwin’s Captain of The Beagle on the journey to South America and The Galapagos.
Fitzroy was a meticulous surveyor, whose work even today, aligns very closely with modern mapping.
He would have been apalled of the way in which real data is altered by value-added techniques. If you value-added Fitzroy’s mapping using this method, The Straits of Magellan would be located in the middle of the Atlantic Ocean! Oh he also ended it all by commiting suicide. It is worth a good read, much more interesting than Darwin.http://www.amazon.co.uk/Fitzroy-Remarkable-Darwins-Invention-Forecast/dp/0755311825/ref=sr_1_3?s=books&ie=UTF8&qid=1282767734&sr=1-3

The project should start with a detailed description of the “signal” it intends to measure. This includes an account of signal characteristics, such as dynamic properties in space and time.

(If we could define the signal in this way, we would have all we need to be able to agree a standard test for statistically significant changes and trends.)

When the signal is understood as set out above, the project can then move onto the specification of a sampling system. Sampling must capture enough data for accurate re-construction of the signal shape from the sampled data.

(Reconstructing the shape of a signal from sampled data is much more demanding than the data required to estimate statistical aggregates.)

The sampling system must satisfy the requirements of the Sampling Theorem in space and time. In practice, data sampling will need to be at least 10 times the theoretical minimum.

I suspect the project has set itself an unachievable objective if it is setting out to measure a global average surface temperature (whatever that may mean).

The cost and effort of data sampling will be too great, and not enough information will be collected. All we’re gonna get is sub-standard sampling and data totally ruined by aliasing. That’s not gonna be worth the effort.

A few long-dated series at rural sites would have greater utility because we can at least understand what the data does and doen’t represent.

I disagree with your analysis. This is no white flag from the Met Office it is a gigantic piece of smoke and mirrors concealing their real trick which ensures that only true believers can get to the data.

If you go back in time just 9 months the Met Office web site offered downloads of nearly raw, up to the last month, surface station data. To be exact you had to select the data then copy and paste in to a file and then in to Excel so it was difficult to do in bulk but it was there and freely available.

There was even a statement on their site that talked about why the historic raw data had been destroyed. Apparently it was technically too difficult and expensive to keep back in the 1980’s (even though a whole world set of data would fit on about 5 tapes).

These data included the stations in Australia and New Zealand and crucial records such as Oxford and Armargh and contained Tmax, Tmin, hours of sunshine and precipitation by month.

Bulk temperature data for a whole rake of countries could be downloaded. It was a bit tedious because you had to open each file and figure out what it one contained but it was there.

The purpose of propaganda is not to tell people what to think, it is to provide an excuse to not think. This statement by the Met Office is designed for that purpose. It uses a lot of words to avoid saying anything. This is the standard MO of the Met Office.

The people: “You said it would be a BBQ summer.”
Met Office: “We didn’t say that.”

That particular game changed with last winter. Last winter the complaint was not over what they said, but what they didn’t say.

The people: “It was the worst winter for 40 years and you didn’t tell us.”
Met Office: “We didn’t say that.”
The people: “We know, that’s what we’re complaining about!”
Met Office: “We didn’t say tha…Oh…well stuff you, we’re not going to give medium term forcasts any more.”
The people: “The ones wrong 6 times out of 6? Good.”
Met Office: “The year 2010 is going to be the hottest ever.”

This data “re-alignment” is going to be interesting because the political background has changed. Not only are they up for sale, but the new government has taken many of the Green projects they supported in opposition out back behind the woodpile. Whether or not the Met Office has taken to heart the game has changed remains to be seen though.

I’m sceptical that this represents a genuine effort to change, rather than a new opportunity to get the same result and say “I told you so”. We have entered a period of Post Modern science whereby only the science that supports the political and social agenda is “good” science. Just as Post Modern history rewrites the historical narrative to suit current agendas, scientific data can (and perhaps must) be rewritten to suit the current political narrative. The Met Office will chase the funding to provide whatever outcome their political masters want.

kuhnkat you need to read the history of the project. the way people write about it they seem to imagine that NOAA was collecting data and then stopped collecting it in 1990 on some stations. That’s just factually wrong. But if you want to know the history just read the documents or do an FOIA. My last FOIA to NOAA arrived in the mail a bit a ago. about 400 pages on the decision to take some metadata down off the web. I will start to take people more seriously when they actually engage in the process of seeking the truth. That means reading the documents that are available, spending more than 5 minutes on a problem, actually filing some FOIA. FWIW.

Here’s a (somewhat paranoid) thought: If a climate model was really useful in predicting climate change, it would have serious military implications–extremely serious. If you understand how climate works, chances are you also understand a lot about how to change it. Do we really want Islamic nutcases and other potential enemies knowing that sort of thing? Wouldn’t it be logical to make sure any non-classified models are run by second-raters and are going down blind alleys? That would explain a lot.

There would almost have to be classified military efforts to understand climate, if for no other reason than to understand what potential enemies might try in terms of altering it. I suspect that it would be focused primarily on regional impacts, with a secondary effort at understanding how those regional impacts interact–a bottom-up approach rather than a top-down one, though that approach isn’t necessary to the idea .

The military would certainly have the resources to have more data on worldwide climate conditions than is publicly available. With all of the Cold War submarine cat and mouse games the military almost certainly knows more about the temperatures of different levels of the oceans than are publicly available. They certainly did have rather advanced weather prediction capabilities for the time as far back as World War II. That doesn’t necessarily give them climate prediction capabilities, but that would be a logical extension of weather prediction efforts.

Climate is scary-complex, and I’m not sure even with a lot more data and competent people in charge of the effort the military could come up with something militarily useful. And of course playing with climate alteration is an extremely dangerous game. I would be very surprised if the military didn’t have both offensive and defensive strategies at least being talked about, modeled far more competently than the publicly available models can model them, and maybe tested on a small scale though.

“If we take advice from McKitrick for starters, and work on the cumulative distortions of basic records from UHI etc, we seem to be looking at 0.5 deg C overshoot at least.”

I believe ross claims that AS MUCH AS 50% of the warming seen ON LAND since 1980 is explained by a function that includes ‘literacy’ in 1979 (LIT79 ) as a predictor and GISS population figures ( 1990 figures that are not very accurate), national coal production….go read the regression equation, just to double check me ok?

“The correlations are quite robust to the inclusion of atmospheric circulation indicators, confirming the presence of significant extraneous signals in surface climate data on a scale sufficient to account for about half the observed upward trend over land since 1980.”

” A substantial fraction of the post-1980 trends in gridded climate data over land are likely not “climatically real” but result from data quality problems and local environmental modifications.”

Lets see. GISS 1980 is about .2 and now about .7.. heck make it .6C warming over land.

Then You go do the math. the land is 30% of the total. Ross claims that AS MUCH AS 50% of the rise seen since 1980 an be explained by a function that uses the variables I described above. So if the rise over land was .6C from 1980, then.. .3C of it can be explained by a function that uses coal production of the country, the literacy rate in 1979, the population in 1990, and whether or not its in the soviet union, and a dummy variable for proximity to water ( coastal warms more slowly) and cosine of latitude. I would not call that UHI. UHI is caused by physical changes to the surface.

In any case, you end up with maybe .15C of the TOTAL global warming that can be explained by the regression equation. (Want to see how that regression works as a prediction? It cant. or whats the literacy rate in 1979 got to do with UHI in 2030?)

In any case. I like Ross’s work because it shows just how SMALL the possible contribution of UHI is.

I haven’t heard of a general house cleaning at the top of The Met, I do notice a rather new PR Meister (Alan Shepherd, Strategic Marketing & Product Director) but can’t imagine that this is anything more than a diversion by the “Kid” to throw the warmists off their scent and get a little breather. The proof is in da’Puddin.

Perhaps homogenization would not cause the hairs to go up on the back of your neck if you actually read the papers.

Homogenization is one approach ( not required) in a chain of analysis.

DETECTING OUTLIERS is a form of homogenization if you decide to throw out outliers. or you can keep them. When an instrument changes you have several choices. One choice is homogenization. Today everybody uses different methods. the methods are tested against “held out” data and assses for their statistical properties. It’s rocket science.. not really. Pretty standard approach.