Thursday, January 28, 2010

It is surprising that such a simple program could have a programming error. But there was a memory overflow, and it had a big effect on the results.

With that corrected, there is indeed a rise in station mean temps. The new graph is here. I should emphasise that the plot is of "naive" means - just averaging all readings for each year, including duplicates.

The new plot is like that on p 14 of the d'Aleo/Watts report, and different from that on p 11.

The new report of d'Aleo and Watts, trumpeting calculations of E.M.Smith, makes much of a supposed shift of GHCN stations to warmer areas as an alleged source of warming. Indeed, it is full of accusations that this is done with fraudulent intent.

Of course, anomaly calculations wouldn't show warming for that reason. But is the station set actually warming?

I did a simple calculation. Just the average temperature of all stations in the GHCN set v2.mean, for any year. You might expect a small rise reflecting global warming. But if there is nett movement of stations to warmer climes, that should show as a bigger effect.

Update 29 Jan. I investigated the downspike about 2006. It's caused by some stations returning a lot of missing months, which yields erratic results. So I put a screen in the program requiring at least 9 months of data before a station could contribute to the year's average. It didn't change the overall picture, but did eliminate the spike at 2006.

Following a suggestion of Carrot Eater, I checked the v2.mean adjusted file. The results, not very different, are here .

Zeke, with that number of stations and without spatial averaging, I'm a little surprised that second plot came out looking so good. I'm feeling too lazy to look myself; I'm wondering what the distribution is between NH and SH for the high latitude discontinuous subset. Polar amplification is more obvious in the NH, after all.

Carrot Eater: here you go. The first number is # northern stations, the second is # southern, all in the discontinuous set (e.g. cover 1960-70 but stop reporting between 1970 and 2000) and >= 60 degrees latitude.

Thank you zeke. I guess that's about what I expected. If the balance between NH and SH swung more rapidly over time, I'd expect some minor spurious trends in your simple mean due to the NH warming more strongly. But the hemisphere distribution looks stable enough to avoid that.

I'm surprised your method works as well as it does. I'm trying to think of an easier alternate way, short of calling up the individual countries and asking them to send in data for the stations they don't send monthly CLIMAT reports for.

I see the Watts/d'Aleo/Smith publication actually does produce the meaningless graph above, though they get a very different-looking result. They don't number their figures, but it's on page 11. Any idea what they did differently?

Interestingly, they cite a rather dated paper from Willmott (1991) on possible artifacts in spatial averages. This also works with absolute temperatures, instead of anomalies. I'll have to take a closer look to see what those authors did.

I'm glad to find this blog. I think a blog that is mainly about the Watts Surface Stations project is very useful, especially as Watts and his moderators say there is a lot more coming.

Now, WUWT has made me all excited before about stuff finally driving that last nail in AGW's coffin home, but up till now every attempt has disappointed me greatly (I of course hope AGW really is a scam and would welcome any conclusive evidence, which the molehills camouflaged as mountains obviously are not).

Would it be a good idea to give the blog a more prominent name, such as StationAudit or you name it? It'd be good if something is already there when the Pielke/Aleo/Watts paper comes out that can evaluate its strengths and weaknesses. People like me would immediately know where to look (instead of waiting for all the scattered blogs and websites to respond) for a review.

OTOH, it will be a lot of work. I don't want to be one of those people who asks other people to carry out his ideas. If I'd have the brains and the expertise for it I would do it myself. :-)

Carrot Eater,Thanks, I missed that. I don't know for sure what the cause of the difference is. The source seems to be Ross McKitrick, as quoted, and he gives an Excel spreadsheet, which I'm looking into. He doesn't use v2.mean, but some data scraped by Joe d'Aleo from the NOAA site. It may be adjusted data. I'm not sure if it's available to others. He then weights categories (urban, suburban and rural) in an unexplained way.

Carrot Eater,I've looked at Ross' Excel file, and tried to track down the source. For some reason, the Excel file starts from data averaged over the three categories, which is why he then creates a weighted sum in putting them back together. There's no actual station data in the file - just averages over these three categories. This makes me think that maybe those averages were what he had got from Joe - otherwise I don't see why the roundabout process of recombining to get the all-stations average.

I tried to track the NOAA refence given, but it starts at this general level :http://www.ncdc.noaa.gov/cgi-bin/res40.pl?page=ghcn.html. When I go further, looking for data that could be found as described, the only plausible page seems to be http://www.ncdc.noaa.gov/cgi-bin/res40.pl?page=ghcn.html, which tells me:Global Climate at a Glanceis undergoing maintenance to improve its function and update its data feed. A timeline for its return will be posted on this page.Last Updated Monday, 26-Oct-2009

Carrot Eater,I had the data handy, so I did that. The trend for the adjusted data is still down. The homogeneity adjustment does seem to level things a bit, and there is a more prominent uptick at about 1990, but it's temporary. The plot is here.

Neven,Thanks for your comments. On blog names, my choice was really to find something that was short and easily searched. I was surprised how many blog names on blogspot were already taken. But is it really so that people find blogs by searching likely names?

One of my motivations is indeed to have a resource for responding in some technical detail to stories that arise on WUWT etc. I'd be very glad of any help there.

just leads me (eventually) to the main data page of the GHCN, http://www.ncdc.noaa.gov/oa/climate/ghcn-monthly/index.php

It looks like the graph was made in 2000. As of now, I don't think there are separate files for urban/suburban/rural; you have to cross-reference v2.mean against v2.temperature.inv to find that. Maybe things were different back then.

jae,One could put that as what caused the dip in the early 80's. But yes, it could be a blip associated with the changeover from the historical data to the recurrent reporting. But it isn't maintained.

Carrot Eater,A curiosity here is that there are plots of apparently the same numbers on p 11 and p 14 of the report. But both the temperatures and the numbers of stations are different (and the temperatures different to mine). The first plot has about twice as many stations.

In any case, I really don't care why your plot is different from p11, or why that is different from p14.

Basically, if one guy divides 5/0 and gets 45, and another guy divides 5/0 and gets 98, it isn't interesting to me how they got those numbers; it's simply enough to know that they shouldn't have divided by zero in the first place.

Most of what Watts/d'Aleo/Smith are presenting are just elementary misunderstandings of anomalies, baselines, trends and the history of the GHCN. It's better to focus on those directly.

TB,Thanks, those are good maps. The post-2008 one is especially helpful. It's clear that the big reduction was in the US, which goes from highly oversampled to just OK. But this may be a USHCN v2 issue.

Placing the dots on a map woth country boundaries might be interesting. Thgere's clearly a lot of countries in Africa that don't report at all (as well as Bolivia).

I think your map is fantastic. That is value added, beyond most other efforts I've seen, including my own. I agree a map with national borders would be good.

Bolivia is small enough that I don't care much. It's more the empty spot in (Angola/DR Congo?) that is of concern to me. Though I wouldn't expect much data to come from DRCongo, given the state of the place.

Could you do a version of the map that gives an idea of stations that give reasonably high frequency of reports? Meaning, can you show a map for stations that have given at least 7 months in each of the last 30 years, or something like that?

I changed the comment rules to allow anonymous comments. I ran the query you suggested and it shows a gap in Africa as you suspected. Canada is also showing a gap too. I think a better kind of graphic is needed to visualize changes over time. A map can't really show which months and years the stations are missing data.