Tuesday, December 9, 2014

What has changed since the version 1 release of the Databank?

It has been nearly six months since we have released the first version of the databank. While this was a big achievement for the International Surface Temperature Initiative, our work is not done. We have taken on many different tasks since the release, and a brief description is below:

Monthly Update System
As described in this post, we have implemented a monthly update system appending near real time (NRT) data into the databank. On the 5th of each month 4 sources (ghcnd, climat-ncdc, climat-uk, mcdw-unpublished) update their Stage 1 data, and on the 11th, their common formatted data (Stage 2) are then updated. In addition, an algorithm is applied appending new data to the recommended merge, and that is updated on the 11th as well.

Bug Fixes
Users have submitted some minor issues with version 1. Some stations in Serbia were given a country code of "RB" when they should have been given "RI." These have been addressed, and a new version of the databank (v1.0.1) was released.

There have been concerns about how the station name is displayed. Non-ASCII characters pose problems with some text interpreters. A module has been created in the Stage 1 to Stage 2 conversion scripts where these characters are either changed or removed to avoid this problem in the future.

Of course issues could still exist, if you find any please let us know! As an open and transparent initiative, we encourage constructive criticism and will apply any reasonable suggestions to future versions.

New Sources
We have acquired new sources that will be added as Stage 1 and Stage 2 data soon, including

300 UK Stations from the Met Office

German data released by DWD

EPA's Oregon Crest to Coast Dataset

LCA&D: Latin American Climate Assessment and Dataset

Daily Chinese Data

NCAR Surface Libraries

Stations from Meteomet project

Libya Stations sent by their NMS

C3/EURO4M Stations

Additional Digitized Stations from the University of Giessen

Homogenized Iranian Data

It is not too late to submit new data. If you have a lead on sources please let us know at data.submission@surfacetemperatures.org. We will freeze the sources again on February 28th, 2015, in order to work on the next version of the merge.