Thursday, May 20, 2010

Dude, where is my thermometer?

Update: The code I used to generate the frames for the animation is now available under the GPL.

The temperature data we have now is not the output of a well-designed measurement process. It could not have been. Throughout time, humans measured and recorded temperatures and weather conditions where they lived and where they traveled.

The GHCN databasehas been employed in several international climate assessments, including the Intergovernmental Panel on Climate Change 4th Assessment Report, the Arctic Climate Impact Assessment, and the State of the Climate report published annually by the Bulletin of the American Meteorological Society.

This time, I decided to not worry about the actual measurements, but just lay out the locations of temperature stations with data in the unadjusted mean file v2.mean.Z every month in the years spanned by the GHCNv2 database.

The result of that effort is the following animation:

Only you can prevent thermometer suicides

From time to time, I and many other more prominent commentators (including, but not limited to, Steve McIntyre, Anthony Watts, E.M. Smith) have noted the disappearance of thermometers from the main temperature data sets.

In case you miss it in the animation, here is what we are talking about:

Temperature stations in GHCNv2 (January 1960)

Temperature stations in GHCNv2 (January 1970)

Temperature stations in GHCNv2 (January 1980)

Temperature stations in GHCNv2 (January 1990)

Temperature stations in GHCNv2 (January 2000)

Temperature stations in GHCNv2 (January 2010)

23 comments:

Simon, thanks. It's a beautiful animation. Forgive me for taking the liberty of slightly repackaging it (to make the 10 second version). I'm sure history books could be written on an analysis of why each zone sprang to life (and collapsed) as it did.http://joannenova.com.au/2010/05/the-great-dying-of-thermometers/Cheers,

Great work Sinan. This is clearer than some animations I have seen. I've just noticed thermometer active from about 1870 in Sri Lanka and only dropped in the last 10 years or so - that is a long record worth looking at. Thank you. Verity

Sinan, I agree about the point of watching the full period from 1701. I enjoyed and stopped the full animation several times to try to piece together what it was that triggered off /that/ pattern of growth. I'd love to see a historian go through it decade by decade...

Systematic temperature recordings are really a marker or proxy for something, I'm just not sure exactly what. Not wealth necessarily, not democracy, not english per se...

As with most things, I think one can blame this on the Turks as well: Ottoman domination in the Mediterranean and the Near East caused Europeans to discover alternative trade routes and expand westward from Europe.

Clearly, the seafaring nations of Europe, the British, the Dutch, the Portuguese and others did much to carry their technology and ways of doing things to various corners of the world.

This is all very tongue-in-cheek, of course ;-)

I do think the English have a lot to do with it. It takes some considerable degree of discipline, organization, entrepreneurial spirit and a whole bunch of other admirable qualities for people to sail across uncharted oceans to far flung places and be able to have the feat repeated by others over and over and over again.

It is really hard to relate to what that required today given all the GPS stuff in our lives.

Trade is wealth. Trade requires reliable routes and weather information. There is sufficient incentive to try to use and improve whatever tools can help with that.

@Neven: My answers to both your questions is: I do not know. In their Peterson &Rose say: "The reasons why the number of stations in GHCN drop off inrecent years are because some of GHCN’s source datasets are retroactive datacompilations (e.g., World Weather Records) and other data sources were createdor exchanged years ago. That does not mean one cannot update them. Also, thatdoes not explainwhy so many stations' data are missing post mid-2005 or so.

As for "the trend", I am not a fan of talking about linear trendlines fittedto small known segments of a very long unknown time series.

"Why are there currently fewer stations in the temperature record? The physical number of weather stations that are reporting temperature data has diminished - some of the older outposts are no longer accessible in real time (NOAA). In fact, the perceived "drop-off" is exacerbated by the fact that the NOAA have been actively adding historical data into the GCHN database from older weather stations that are no longer active, in an attempt to provide more comprehensive coverage of the past."

"It’s common to think of temperature stations as modern Internet-linked operations that instantly report temperature readings to readily accessible databases, but that is not particularly accurate for stations outside of the United States and Western Europe. For many of the world’s stations, observations are still taken and recorded by hand, and assembling and digitizing records from thousands of stations worldwide is burdensome.

During that spike in station counts in the 1970s, those stations were not actively reporting to some central repository. Rather, those records were collected years and decades later through painstaking work by researchers. It is quite likely that, a decade or two from now, the number of stations available for the 1990s and 2000s will exceed the 6,000-station peak reached in the 1970s."

The way I understand it some time or other a lot of the dropped stations will be updated.

In the meantime a lot of bloggers (alarmists as well as skeptics) seem to have been building their own temperature reconstructions to compare them to GISS, NOAA and NCDC.

All of them find the station drop-out is not a source for bias. And that's the most important thing, right?

"The physical number of weather stations has shrunk as modern technology improved and some of the older outposts were no longer accessible in real time.However, over time, the data record for surface temperatures has actually grown, thanks to the digitization of historical books and logs, as well as international data contributions. The 1,500 real-time stations that we rely on today are in locations where NOAA scientists can access information on the 8th of each month. Scientists use that data, as well as ocean temperature data collected by a constantly expanding number of buoys and ships – 71 percent of the world is covered by oceans, after all – to determine the global temperature record."

Have you ever considered approaching NOAA or GHCN with your questions? I'm sure they have people who are willing to answer your questions (like "why so many stations' data are missing post mid-2005 or so").

@Neven: Have you ever looked at the documentation for the Health and Retirement Study? Or, BRFSS? These are data sets that are orders of magnitude more complicated than the GHCNv2 and you can find out pretty much any question regarding the data set by reading the documentation.

The answer to your questions: No, I haven't contacted them. Nor should we have to contact them individually for such basic information about the data set.

Finally, I am not able to think of a good reason why Central Park, NY would no longer contribute to the GHCNv2.

"Can we hold the press releases comparing today's temperatures to the past century until then?"

Like I said, various bloggers have shown that the dropping of weather stations hasn't influenced the temperature trend in any meaningful way. So yes, we can compare today's temperatures to the temperatures of past century. And it's a useful thing to do, because this data tells us something about the amount of energy, trapped by greenhouse gases, that is in the atmosphere.

In my opinion it would be unwise to wait a few decades, especially as the filling of remaining gaps will probably not significantly alter the results we already have.

@Neven: We'll have to agree to disagree then: I do not understand how one can show how some variable whose values we do not know would not have affected an average calculated using the variables whose values we do know.

What strikes me is how many of the stations in Canada & Alaska suddenly disappear in later years, as if the data-collection stopped at the northern USA border. And wouldn't those many stations tend to be 'cooler'? To be fair, I note Australia pretty-much disappears as well.