Hurricane Data Compilation Thread

UKweatherworld has a thread where people don’t comment but simply contribute references on the Holocene Optimum. I’m going to start doing things like this as a way of keeping track of things. People have contributed data but it’s impossible to keep track of. This thread is going to consist only of scripts and data relevant to replicating the hurricane papers. Any general commentary will be deleted. I’ll put some of the data sets into *.txt files so that they can be called. Please provide references to some of the data sets.

32 Comments

Here’s a script which gives the location of Kaplan’s SST data. It’s in Netcdf format. Fortunately there’s an R package which reads this format. I can’t say that I know my way around the format but I was able to almost instantly recover the relevant data and make average temperatures for the 6 Webster et al basins.

Here’s the Atlantic PDI data as collated by Willis including David Smith’s update for 2005-1006, as well as SST up to 2004 (I’ll update this.)
Data http://data.climateaudit.org/data/hurricane/pdi.atlantic.txt
See: http://www.climateaudit.org/?p=803#comment-46813
bender adds: The “PDI” values are from Emanuel (2005) and so are the “SST” values (=”RawHadCRUT_SST”). The SSTs must be anomalies since they are in the range [-1,1]. HadISST are sometimes used in these analyses (e.g. that’s what Landsea has used) and these appear to be actual temperatures as
they are in the range [26,30]

Well, all the Kaplan SST data sets look very normal in a Histogram plot, and so do the residuals of a linear fit (unsurprisingly). I tested the NATL, NIO, and SWP sets for autocorrelation, and they all show ~0.9 at lag 1 (p

Darn. :-) I’ll do the analysis again tomorrow. It just showed deep auto- and intercorrelation in the Kaplan SST’s, even across data sets representing physically distant locales. If that’s not interesting, let me know.

Webster et al in their Response to Comment (Science 311, March 2006) stated that Chan Science 2006 and Emanuel 2005 used the following data sets to construct their PDI:http://www.weather.unisys.com/hurricane/index.html
which provides information for individual storms in 6 basins for the following periods:
ATL 1851-2005
EPAC – 1949-2005
WPAC 1945-2005
SPAC only 2000-2005 as far as I can tell so far
SIO 1949-2003
NIO 1945-2003

Emanuel 2005 says:

Owing to changes in measuring and reporting practices since systematic observations of tropical cyclones began in the mid-1940s, there are systematic biases in reported tropical cyclone wind speeds that must be accounted for in analysing trends. The sources of these biases and corrections made to account for them are described in Supplementary Methods.

Emanuel’s SI describes the adjustment but does not present the unadjested data. Landsea 2006 presents different results for Atlantic PDI. Is this data archived anywhere?

What collations of the data are available other than what Willis obtained in his email from Emanuel?

Here are a few of the links I contributed to UKweatherworld just searching around NASA. The last link though is not a technical paper; however, it provides a pretty good description of the tropical geology of SE USA from about 7K years ago. It also has some strong references WRT geological and population characteristics and these are reinforced by strong; but, aged studies. If these are of use feel free to add if not it is not an issue.

I would like to submit these links as sources of data that can help explain recent water vapor research that participates in hurricane formation. I did my best to make sure to include links that should terminate in professional studies from a .gov site.

Note that this is specific to the Denver CO area; however, the characteristics on 7/26, 8/14, 9/21-22, 10/1 2006 seem to be indicitive of an unusual characteristic of a dual direction of freedom from a single phenomon. (It has to be related to the depth of measeurement penetration rather then be a dual vector of change from a single phenomon.)

The ZIP file contains the full database as an ASCII text file, and also contains a text file explaining the format.

The information is in lines of text for 6 hourly data that includes the BoM season and cyclone number, cyclone name, data and time, lat and lon, and various meteorological factors including central pressure, maximum winds, wind radii, etc.. A spreadsheet can easily be used to separate out whatever parts you may be interested in.

Researchers should note that the BoM uses the WMO standard 10-min-av winds instead of the 1-min-av winds used in the US, and this issue will need to be accounted for if comparing to say US best track data for the same cyclones – this can also be an issue when using independent data for other basins as well.

Further to this, those unfamiliar with the BoM need to know that when making direct comparisons, much of the modern US best track material is largely satellite based, while in addition to satellite data the BoM uses a wide range of material from both land based instrumentation including radar (where appropriate) and shipping reports, and that extensive post-storm analysis is done for many landfalling systems which may all be taken into account in ‘best track’ data.

The total number of cyclones in any season contained in the US and BoM data sets can be different due to both the amount and type of data taken into account, and different definitions of what a cyclone is – the BoM uses 10-min-av winds greater than 33 knots in all 4 quadrants, while the US uses 1-min-av winds greater than 33 knots in any quadrant.

The cyclone intensity information can vary significantly between the BoM and US data – in most cases the BoM data is more accurate due to both local experience and the sheer volume of data used in the analysis.

The full BoM database was undergoing a quality control revision in 1999-2000, however I understand the process is still incomplete (due to funding cuts, if my memory serves me correctly).

Re: #22 – Steve, any JTWC 80+ numbers in best tracks data are those systems that were not identified as storms in real time, but were later found to be storms when re-analysing the data.

This happens from time to time with marginal systems, mainly due to the pressures of getting information out at regular intervals with only a small number of mets working at any particular time, as the poor satellite imagery presentation of many maginal systems combined with lack of other supporting data means it can take a pro-met many hours of careful analysis of multiple satellite images before making a judgement that it is in fact a storm, and time may not be available to do this process properly until after the season when the best tracks are being put together.

Most marginal systems that fall between the cracks in real time are a long way from inhabited areas where other data can help making such a judgement, and in real time they are usually being tracked as ‘invest’ systems that have potential to develop and are assigned 90+ numbers, so if anyone in a boat or plane is intending to be in the vicinity they can usually get enough info to see that the weather could deteriorate in the area.

Re the hurricane data quality issue. I am becoming increasingly convinced that the data outside the NATL is problematic. I suggest a two pronged approach. Prong 1: use exactly the same data as Emanuel to assess what inferences can actually be made from the time series of SST and PDI in NATL and WPAC from 1949-2005 (and ignore data quality issues, just focus on auditing the statistical methods). Prong 2: use the NATL data record (HURDAT) since 1851 to see what can be determined from the longer time series with varying levels of uncertainty as you go back in time. If you agree with this approach, then the data issues become: making sure you have the same data as used by Emanuel, and then sorting through the NATL data quality issues. sorting through multiple datasets, many of dubious quality, could be huge time sink and not really get us anywhere. Lets have some discussion about what data sets we are going to look at, before digging into the various warts associated with a number of different data sets.

#25. I’m not planning to get into hurricante track QC issues – I don’t have the time; I’m already spending more time on this than I should without previously finishing some proxy papers. I’m just going to work through some of the calculations and see if I can repeat them.

There is extensive information on individual West Pacific storms from old annual reports of the Joint Typhoon operation, now online. An example is here https://metocph.nmci.navy.mil/jtwc/atcr/1959atcr/pdf/section5.pdf. There’s a lot of data on individual storm tracks that could be easily digitized with a little clerical work.

A well-written article on tropical atmospheric circulation is given here . These Chinese researchers studied Hadley cells and associated features. Nice cross-sections of the ITCZ and circulation cells.

My opinion is that some of the mystery about climate shifts will disappear as we learn more about the behavior of these massive heat removal systems. The Southern and Northern circulations have the same general features but differ in ways (see diagrams) that might affect their ability to remove heat. Perhaps at times one hemisphere’s Hadleys and Ferrels do a better job of removing heat than does the other.

As a side note, if I remember correctly, Tim Ball mentioned that the GCMs do a rather poor job of modeling these important circulations.