Why European forecasters saw Sandy’s path first

US weather model is good, but lags behind the best.

Look, it’s not easy to predict the weather. And while it’s frustrating to have weekend plans spoiled by unexpected showers, the stakes are much higher for potentially catastrophic storms like hurricanes. The warning provided ahead of “Superstorm” Sandy’s arrival on the East Coast saved lives, without question, and that’s a success for weather science. But the story carries a reminder that there’s room for improvement in the US, where many argue forecasting tools have been neglected amid continual budget crunches.

Seven days before Sandy made landfall in New Jersey, the atmospheric crystal ball was partly cloudy. The US National Weather Service forecast model showed a chance that Sandy might come ashore, but indicated that it was more likely the storm would spin off into the Atlantic. The European Centre for Medium-Range Weather Forecasts (ECMWF) model, however, definitely pointed the storm ashore. It would be about three days before the US model totally converged on the Europeans' forecast.

The fact that the National Weather Service was able to issue detailed warnings well ahead of Sandy is both a testament to what these amazing models are able to do, and the hard work of the professionals behind the scenes.

Still, some in the US couldn’t help but wonder why the European model beat its American counterpart to the punch in predicting Sandy’s landward turn. Did it get lucky or is it a superior model? While it doesn’t always beat the competition (it got 2012’s Tropical Storm Debby wrong, for example), most scientists agree that the European model holds a clear advantage. Sandy just brought that fact before the public eye.

Weather wizard needs food, badly

Like any suburbanite who finds themselves coveting a neighbor’s flashy new gizmo, we first need to understand what, exactly, that gizmo is. Forecasting models are computer simulations of the global atmosphere. They’re similar to the models used by climate scientists, but with fewer components—leaving out slowly-changing factors like ocean circulation, vegetation, and atmospheric gas concentrations. Forecast models can also be run at a higher resolution that can better simulate local-scale weather. Data from around the globe is fed into the model, which then simulates the consequences of atmospheric physics into the near future.

Beyond 72 hours, the European model significantly outperformed US forecasts.

Forecast models require some serious computational horsepower, which can only be supplied by supercomputers. The ECMWF, for example, utilizes an IBM system capable of over 600 teraflops that ranks among the most powerful in the world, and it's used specifically for medium-range models That, fundamentally, is the reason their model frequently outperforms the American one. The US National Weather Service’s modeling center runs a diversity of short-, medium-, and long-term models, all on a much smaller supercomputer. The National Weather Service has to do more with less.

This computational bottleneck limits the US model in two key ways. First, it runs at a coarser spatial resolution (about 25 kilometers, as opposed to 15 in the ECMWF model). Anything in the atmosphere that takes place at a smaller scale has to be approximated. In general, finer resolution models can directly simulate more processes, especially once you reach the scale of an individual storm cell.

Second, the way in which measurements are fed into the models differs. The US model takes “snapshots” of data, builds a global picture (the initial conditions), and then begins the forecast simulation. The ECMWF model, on the other hand, takes continuous observations spanning half a day, runs the model with that real data, and then sets it loose on the future. While computationally expensive, this can result in more realistic initial conditions, and is part of the reason why the ECMWF model is usually reliable a couple days further into the future then the US model.

Tomorrow's forecast with yesterday's budget

The US model has lagged behind in these areas for a number of years, but the money needed to push ahead has never shaken loose. The National Centers for Environmental Prediction did get a new building this year, and an upgrade to the supercomputer is scheduled for August 2013. Unfortunately, that upgrade won’t be nearly enough to begin catching up to the ECMWF. Cliff Mass, a professor of Atmospheric Sciences at the University of Washington, told Ars the new system would be about 50 percent more powerful than the existing one—a far cry from the performance leap he has advocated.

Other than insecurities about failing to be #1, is this a problem for the US? When the National Weather Service issued its warnings ahead of Sandy, the forecasters weren’t limited to American information. They looked at the results from the ECMWF model, as well as the Canadian and UK models. In this way, the models benefit everyone, regardless of where they come from.

But that doesn’t mean the status of the American model is irrelevant. As Mass explained to Ars, the local and regional models used for detailed, short-term forecasting utilize the simulations from the longer-range National Weather Service model. They also share its system for building initial conditions from observed data. So even if the US can lean on the ECMWF model’s forecast to help figure out what’s likely to occur six to eight days down the line, it can't do so for the local forecasts. And these are the ones Americans depend on daily.

The Future of Forecasts

Cliff Mass has argued on his blog that investments in weather forecasting capabilities would pay off handsomely, and that huge advances are waiting on the necessary computational horsepower. One such advance is the ability to run many simulations of a model and generate probability distributions for forecasts. Weather is, to an important degree, a chaotic system. Run a model twice with slightly different initial conditions, and the outcomes can be quite a bit different.

For example, when the US model was run multiple times to project Sandy’s path, some simulations showed the storm making landfall but others sent it out to sea. With a large number of simulations, a calculated probability of landfall would have better characterized the model’s forecast than eyeballing a small sample of potential outcomes. The same is true for all forecast modeling, but this obviously creates a lot more work for the supercomputers behind those models. As a result, this technique is currently not applied at resolutions as fine as the ECMWF probabilistic simulations. And in general, current models are simply too coarse to capture some important small-scale details. Finer resolution would mean better forecasts.

Of course, most agencies have a long list of great things they could do with more funding, but money is limited and budget politics aren’t going anywhere. There are always competing needs, even within US weather programs. Many of the weather satellites that provide critical data for the forecast models are American. Forecast models around the world depend heavily on that data, but a number of the satellites are aging and funding problems are threatening their replacements.

A National Research Council report released in April stated that the “number of NASA and NOAA Earth observing instruments in space by 2020 could be as little as 25 percent of the current number.” The result? “Consequences are likely to include slowing or even reversal of the steady gains in weather forecast accuracy over many years and degradation of the ability to assess and respond to natural hazards and to measure and understand changes in Earth’s climate and life support systems.”

And the forecasters' experience backs this up. After Sandy, the ECMWF ran a simple experiment. They ran their model again with the data available five days before Sandy hit New Jersey, but left out data from NASA’s polar-orbiting satellites. (Delays to the replacements for those satellites mean there’s a real possibility we could lose their capabilities in the interim.) Without that data, the ECMWF model failed to predict Sandy’s landfall.

Maintaining current forecasting capabilities will cost money, and building on that success will cost the US even more. Of course, not spending it also has a cost—the cost of unreliable forecasts (measured in dollars and, perhaps, lives) is one that we all would rather avoid.

81 Reader Comments

Dear US Government,PLEASE get your heads out of your collective asses and invest in our satellite and terrestrial infrastructure. Our roads, bridges, and power grids are failing and in desperate need of updating. Our NOAA and NASA satellites are not perfect, but agencies around the world use the data from them and we are all richer for it.

You can choose to invest in billon dollar military planes that are of questionable usefulness, or you can put the money to good use by investing in our much needed infrastructure. (Oh, and it would create jobs, which, ya know, would be a good thing...)

This was a very enjoyable article to read. I had no idea that the US could have lagged so far behind another forecasting system - and for a mostly fixed cost reason like hardware too. Articles like this should be in all our major papers. Important, accessible, and something that the non-techy can get behind.

So the European model is better because they have more money to spend after freeloading on our data gathering abilities. Sounds kind of like how they have a better social safety net after freeloading under our defense umbrella.

Reference for where they have access to forecasting data for free?

mavaggie08 wrote:

Something like weather forecasting seems like it might be better served by an international alliance, rather than national level weather services. It doesn't make any sense for one nation to foot the majority of the bill, nor does it make any sense for several nations to replicate each others abilities. Maybe we could get the NATO countries together under one weather service? I mean, they already swap calssified defense data, what's a few barometric pressure readings and satellite pictures between allies?

I would stay away from one gargantuan system. There is a benefit to a distributed approach. Different ideas can percolate to the top. It's not like we have three hundred forecasting engines. There are a few around the world.

So the European model is better because they have more money to spend after freeloading on our data gathering abilities. Sounds kind of like how they have a better social safety net after freeloading under our defense umbrella.

So the European model is better because they have more money to spend after freeloading on our data gathering abilities.

Thus allowing you to freeload on their forecasting abilities? First of all, nobody else, with the possible exception of Russia, was close to being able to put those satellites in orbit "back then". Of course, neither is the US these days. Secondly, it seems you are assuming that they are getting that data for free, which I very much doubt.

How about if we check if all the people who are deciding they are mentally ill and therefore need to be on Social Security Disability, just about the same time their 99 weeks run out, are really that crazy. They are over 2% of the population.

We can take the money we save on phony disabilities and redirect it toward upgrading this system. We should have more than enough for this and hundreds of other things.

This is a fantastic idea. I can not think of a better place to take money from than from people who are possibly mentally ill. In addition, since the proposed checking of will require absolutely no additional time or money, it's completely free money.

And yet again we see where politicians crapping all over the NASA budget have real-world impacts. Especially on this, it's potentially a huge negative impact directly on their constituents, not just hypothetical job loses in specific districts, it's across the country, especially on the coasts.

I stopped reading a few lines in... All I could think about was the story of "forecasters" being sued because they failed to warn about a major earthquake. This story sounds like the other side of the coin.

How do you mean? It's clearly not attacking any scientists for getting things wrong or praising scientists for getting things right. It's about comparing the policies of the US and Europe when it comes to funding critical science that saves people's lives.

How about if we check if all the people who are deciding they are mentally ill and therefore need to be on Social Security Disability, just about the same time their 99 weeks run out, are really that crazy. They are over 2% of the population.

We can take the money we save on phony disabilities and redirect it toward upgrading this system. We should have more than enough for this and hundreds of other things.

This is a fantastic idea. I can not think of a better place to take money from than from people who are possibly mentally ill. In addition, since the proposed checking of will require absolutely no additional time or money, it's completely free money.

So the European model is better because they have more money to spend after freeloading on our data gathering abilities. Sounds kind of like how they have a better social safety net after freeloading under our defense umbrella. Something like weather forecasting seems like it might be better served by an international alliance, rather than national level weather services. It doesn't make any sense for one nation to foot the majority of the bill, nor does it make any sense for several nations to replicate each others abilities. Maybe we could get the NATO countries together under one weather service? I mean, they already swap calssified defense data, what's a few barometric pressure readings and satellite pictures between allies?

Obvious troll is obvious.

What's really scary is that 5 dummies felt the need to upvote this tripe. For shame.

This is a non-article. The author clearly misunderstands how NOAA, the National Weather Service, and its sister services worldwide rely on weather forecasting tools.

Firstly, the author is erroneous in referring to the National Weather Service as the provider of tropical weather forecasts for United States. Instead, these forecasts are provided by the National Hurricane Center. The NHC _is_ part of the NWS, but is a largely independent unit. Both of these organizations are part of NOAA.

Secondly, it is accepted by meteorologists and meteorological organizations worldwide that any forecast beyond 72 hours is entirely unreliable. Even the finest-grained model cannot be trusted beyond the three-day mark for any statistical accuracy. Year to year, the successes of different models vary, but they are all common in being entirely unable to predict anything more than broad trends beyond that three day mark. Move out to 96 or 120 hours and the accuracy decay is geometric. This is acceptable for temperature and precipitation local forecasting, but predicting events as massive and complex as tropical cyclones based on such inaccuracy is ridiculous.

Continuing, the EMCWF is _not_ a European-exclusive model. The data from the EMCWF and other European models (GFS and UKMET, among others) is provided to official organizations worldwide. It is simply a tool used along with numerous others provided by universities and national forecasting services. The NHC typically provides updates every six hours during hurricane season, with 3-hour updates for major weather events. Every forecast cycle includes the EMCWF, along with numerous others. I refer you to http://www.nhc.noaa.gov/modelsummary.shtml for a list of models used for official tropical discussions, forecasts, and advisories.

It should be noted that these models are run at different intervals and serve different purposes. The big daddy of models, the GFS, is a UK-based model that runs in 12-hour cycles and is used for long-term forecasting. Other models, notably the RUC, run at very short intervals to provide warnings for rapidly evolving events, such as tornadic weather. The models pertinent to NHC are utilized based on availability at the time of the forecasts, forecast discussions, and public advisory releases.

Continuing the model purposes theme, each model's accuracy varies from year to year, and tropical weather events are built on a consensus of large-scale models and the track record of models from year-to-year. Some years the NHC has to to completely disregard models that fail to accurately make predictions at even 24-hour intervals or place additional weight on recent successes by certain models. The official forecast model by the NHC is actual a consensus model built on numerous models, such as the NOGAPS (US Navy), UKMET (UK Meteorology Office), the EMCWF, and the NWS's own consensus models. From there, a large amount of human intuition and skill enters into the final forecast tracks. These forecasts are always accompanied by an often highly-detailed forecast discussion regarding margins of error, reasoning for predictions, and which models were favored in the final forecast.

As for satellites operated by NOAA: most of them are old, and they are failing. Recently NOAA launched a new polar orbiter, but the replacement rate versus failure rate of satellites is worrisome. The western hemisphere relies on NOAAs satellites, some of which have been in operation for over three decades(!), albeit in highly limited capacity. NOAA keeps a number of backup satellites to move into position once primary satellites fail, but these are dwindling in number and replacements are not being launched fast enough. This is a budget issue for the NESDIS office--which is not part of the NHC or NWS, and unrelated to forecast model budgets. Forecast data feeds from an incredible number of sources, and satellite data is more than just visible-spectrum and infra-red photography. For information on satellite operated by NOAA and their statuses, go here: http://www.osdpd.noaa.gov/ml/svcs/index.html.

To summarize this, with respect to the author, his research into tropical weather forecasting--whether by the NHC or other services worldwide--is woefully lacking and uninformed. The author should have read a primer on the use of forecast models before writing this article. The Wikipedia page for Tropical Cyclone Forecast Model is sufficient and the NHC has excellent public-consumable materials, as well.

So the European model is better because they have more money to spend after freeloading on our data gathering abilities.

Thus allowing you to freeload on their forecasting abilities? First of all, nobody else, with the possible exception of Russia, was close to being able to put those satellites in orbit "back then". Of course, neither is the US these days. Secondly, it seems you are assuming that they are getting that data for free, which I very much doubt.

Mmm, not really so true. Weather satellites managed to beat virtually everything else into service; TIROS-1 was launched in 1960, almost at the same time as the really high priority CORONA spy satellites. The only thing really limiting the Europeans from getting in was their lack of a launch vehicle, and as soon as that was rectified in the late 1970s they seem to have started launching their own, based on Wikipedia.

In any case, the impression given by the article that Europe (or anyone else other than the US) doesn't have weather satellites is false. The Europeans actually have three, the same number as the US, and Japan, Russia, and China all have their own networks. The issue is less that there won't be data without US satellites and more that there will be less data, which is obviously not at all the same thing.

Also, note that Europe and the US in particular, and also Japan, do cooperate very closely in Earth observation...not just weather satellites, but eg. the A-train constellation and a variety of missions with multi-sourced instruments (European instrument on an American satellite, etc.)

In all seriousness - did you read the article? It appears that your comment and the article cover some very similar bases. In addition, the graph in the article shows that the EMCWF model actually has a better track record after 72 hours.

Yes it's difficult, no we can't do it perfectly now, but we can always improve, and if someone has already improved, we need to take a look at how they improved.

We still have a debt problem, and raising taxes while subsequently raising spending doesn't do us any benefit.

Better weather prognoses would cut down on weather related harm and costs. And it has already been argued here that it will likely increase jobs, so another benefit.

So would starting to assuage AGW by increasing energy efficiency et cetera. US, and other nations, can do a lot to decrease any national debts by investing rationally.

I had to look up "entitlement". Seems there is a cultural difference, and an unwarranted tie to narcissism et cetera. [ http://en.wikipedia.org/wiki/Entitlement ] Over here it is called "social security" and it is what differentiates Europa and Asia from dysfunctional societies like, say, US. (Dysfunctional in the statistical sense of Roslings world statistics, I haste to add. There is nothing a priori judgmental in the measure.)

thanks for posting. Very enlightening indeed. Do you agree with the basic premise that the NHC supercomputer is underpowered by modern standards or is this really just an oversimplification too far in what sounds like a very cooperative and complex world of forecasting?

Scott, you explained the ECMWF acronym, but what the heck are HWRF, GFDL and GFS? None of these seem to match the initials for the National Weather Services which is the only American model that you mention, although digging in your links one can find GFS, but what is it and what about the 2 others models in the unexplained chart?

I agree with some of the other comments stating that all these forecast centers seem to be a waste of resources since the data is available globally, but we should probably expect the U.S to be at least one of them at the top. For information, what was the cost of the latest ECMWF Power7 rig, what would have it cost to the U.S. to have one of those? There's even faster Power7+ processors now, and Linux had its first Power8 patches earlier this month. The DARPA contracts won by IBM a few years back to develop teraflop supercomputers were "only" a few hundred million dollars, so it can't cost too much now, especially since all the work was already done for the ECMWF, all you need is purchase the hardware to have one, not design a supercomputer from scratch.

This is a non-article. The author clearly misunderstands how NOAA, the National Weather Service, and its sister services worldwide rely on weather forecasting tools.

Firstly, the author is erroneous in referring to the National Weather Service as the provider of tropical weather forecasts for United States. Instead, these forecasts are provided by the National Hurricane Center. The NHC _is_ part of the NWS, but is a largely independent unit. Both of these organizations are part of NOAA.

Secondly, it is accepted by meteorologists and meteorological organizations worldwide that any forecast beyond 72 hours is entirely unreliable. Even the finest-grained model cannot be trusted beyond the three-day mark for any statistical accuracy. Year to year, the successes of different models vary, but they are all common in being entirely unable to predict anything more than broad trends beyond that three day mark. Move out to 96 or 120 hours and the accuracy decay is geometric. This is acceptable for temperature and precipitation local forecasting, but predicting events as massive and complex as tropical cyclones based on such inaccuracy is ridiculous.

Continuing, the EMCWF is _not_ a European-exclusive model. The data from the EMCWF and other European models (GFS and UKMET, among others) is provided to official organizations worldwide. It is simply a tool used along with numerous others provided by universities and national forecasting services. The NHC typically provides updates every six hours during hurricane season, with 3-hour updates for major weather events. Every forecast cycle includes the EMCWF, along with numerous others. I refer you to http://www.nhc.noaa.gov/modelsummary.shtml for a list of models used for official tropical discussions, forecasts, and advisories.

It should be noted that these models are run at different intervals and serve different purposes. The big daddy of models, the GFS, is a UK-based model that runs in 12-hour cycles and is used for long-term forecasting. Other models, notably the RUC, run at very short intervals to provide warnings for rapidly evolving events, such as tornadic weather. The models pertinent to NHC are utilized based on availability at the time of the forecasts, forecast discussions, and public advisory releases.

Continuing the model purposes theme, each model's accuracy varies from year to year, and tropical weather events are built on a consensus of large-scale models and the track record of models from year-to-year. Some years the NHC has to to completely disregard models that fail to accurately make predictions at even 24-hour intervals or place additional weight on recent successes by certain models. The official forecast model by the NHC is actual a consensus model built on numerous models, such as the NOGAPS (US Navy), UKMET (UK Meteorology Office), the EMCWF, and the NWS's own consensus models. From there, a large amount of human intuition and skill enters into the final forecast tracks. These forecasts are always accompanied by an often highly-detailed forecast discussion regarding margins of error, reasoning for predictions, and which models were favored in the final forecast.

As for satellites operated by NOAA: most of them are old, and they are failing. Recently NOAA launched a new polar orbiter, but the replacement rate versus failure rate of satellites is worrisome. The western hemisphere relies on NOAAs satellites, some of which have been in operation for over three decades(!), albeit in highly limited capacity. NOAA keeps a number of backup satellites to move into position once primary satellites fail, but these are dwindling in number and replacements are not being launched fast enough. This is a budget issue for the NESDIS office--which is not part of the NHC or NWS, and unrelated to forecast model budgets. Forecast data feeds from an incredible number of sources, and satellite data is more than just visible-spectrum and infra-red photography. For information on satellite operated by NOAA and their statuses, go here: http://www.osdpd.noaa.gov/ml/svcs/index.html.

To summarize this, with respect to the author, his research into tropical weather forecasting--whether by the NHC or other services worldwide--is woefully lacking and uninformed. The author should have read a primer on the use of forecast models before writing this article. The Wikipedia page for Tropical Cyclone Forecast Model is sufficient and the NHC has excellent public-consumable materials, as well.

With respect, I don't think you've identified any errors in what I've written.

In any case, the impression given by the article that Europe (or anyone else other than the US) doesn't have weather satellites is false. The Europeans actually have three, the same number as the US, and Japan, Russia, and China all have their own networks. The issue is less that there won't be data without US satellites and more that there will be less data, which is obviously not at all the same thing.

When you say Europe has three, you just mean polar orbiting satellites? I'm assuming the Meteosat satellites are still up there/functioning?

[Bear in mind my last experience of weather satellites was using a BBC Micro as a framestore to decode satellite transmissions in the school computer labs, about 20 years ago - so this is a genuine question :-).]

We still have a debt problem, and raising taxes while subsequently raising spending doesn't do us any benefit.

Better weather prognoses would cut down on weather related harm and costs. And it has already been argued here that it will likely increase jobs, so another benefit.

So would starting to assuage AGW by increasing energy efficiency et cetera. US, and other nations, can do a lot to decrease any national debts by investing rationally.

I had to look up "entitlement". Seems there is a cultural difference, and an unwarranted tie to narcissism et cetera. [ http://en.wikipedia.org/wiki/Entitlement ] Over here it is called "social security" and it is what differentiates Europa and Asia from dysfunctional societies like, say, US. (Dysfunctional in the statistical sense of Roslings world statistics, I haste to add. There is nothing a priori judgmental in the measure.)

It would cut down on weather related harm and costs? I'm not sure how. We typically know of severe weather days ahead of time. Even with our "inferior" modeling. So I don't see how this will do anything on the saving lives front. If that's your goal better evacuation and supply procedure is the issue at hand.

As far as costs just because you can predict a hurricane coming quicker doesn't mean you're going to prevent much or any of the damage.

Increasing government jobs does nothing for reducing debt. The government is not a jobs program lets stop using this as a rationale.

Scott, you explained the ECMWF acronym, but what the heck are HWRF, GFDL and GFS? None of these seem to match the initials for the National Weather Services which is the only American model that you mention, although digging in your links one can find GFS, but what is it and what about the 2 others models in the unexplained chart?

I tried desperately to avoid the sea of acronyms. GFDL stands for NOAA's Geophysical Fluid Dynamics Laboratory, and HWRF is another NOAA- Hurricane Weather Research and Forecasting model. There are a lot of other models that could be mentioned (UK, Canadian, etc.). That graph was just one illustrative example of the kind of thing being discussed. (Click on the source link to see where it came from: http://www.srh.noaa.gov/mfl/?n=sandy)

Quote:

I agree with some of the other comments stating that all these forecast centers seem to be a waste of resources since the data is available globally, but we should probably expect the U.S to be at least one of them at the top. For information, what was the cost of the latest ECMWF Power7 rig, what would have it cost to the U.S. to have one of those? There's even faster Power7+ processors now, and Linux had its first Power8 patches earlier this month. The DARPA contracts won by IBM a few years back to develop teraflop supercomputers were "only" a few hundred million dollars, so it can't cost too much now, especially since all the work was already done for the ECMWF, all you need is purchase the hardware to have one, not design a supercomputer from scratch.

I wonder what the NOAA and NASA could do with the budget of the TSA. They also could probably build the next generation of satellites and a new super computer made out of gold with 1/10th the budget of the F-35 boondoggle.

In all seriousness - did you read the article? It appears that your comment and the article cover some very similar bases. In addition, the graph in the article shows that the EMCWF model actually has a better track record after 72 hours.

Yes it's difficult, no we can't do it perfectly now, but we can always improve, and if someone has already improved, we need to take a look at how they improved.

I did read the article. The graph for accuracy displayed is remarkably useless, as it only shows the accuracy in models for one tropical cyclone. And again, after 72 hours is wildly inaccurate and cannot be relied upon, regardless of the model. Stating that a better track record after 72 hours from one model compared to another irrelevant--the models are still too inaccurate for actual forecasting. Note that the EMCWF was still off by 100 nautical miles.

It would cut down on weather related harm and costs? I'm not sure how. We typically know of severe weather days ahead of time. Even with our "inferior" modeling. So I don't see how this will do anything on the saving lives front. If that's your goal better evacuation and supply procedure is the issue at hand.

As far as costs just because you can predict a hurricane coming quicker doesn't mean you're going to prevent much or any of the damage.

Increasing government jobs does nothing for reducing debt. The government is not a jobs program lets stop using this as a rationale.

Evacuations are expensive and both time- and resource-intensive. Better forecasting allows for more efficient triage of these inherently limited commodities, and potentially reduces panicky over-reaction in other areas -- note that the economic cost of a natural disaster includes lost productivity.

We are crossing acronyms, with my apologies. There is a British GFS and an American GFS.

There are no outright errors in your article, but rather a very misleading theme--namely, that forecasts beyond 72 hours by the EMCWF are usable in hurricane forecasting. They are not, nor are the forecasts of any other model presently available. Models improve regularly, but the 72-hour mark remains a barrier to forecasters. A relatively lower level of inaccuracy by the EMCWF in 96+ hour forecasting is not an absolute measure of its accuracy--it's just less wrong than the other models available.

In all seriousness - did you read the article? It appears that your comment and the article cover some very similar bases. In addition, the graph in the article shows that the EMCWF model actually has a better track record after 72 hours.

In the case of one singular example. Probability dictates that any given model will be accurate for some hurricane over an arbitrary period of time. Or, in other words, even if a model is completely broken, you could still find an example where it is more accurate than even the best model.

I'd like to see a graph of how accurate the US model is over time, on average, compared to how the European model fares. As it stands, that graph alone is utterly meaningless. I'm sure the higher resolution makes the European model better, but I'm curious just how much better it really is.

I don't work for NOAA, but I do specialize in HPC and have some inside experience with them.

This article somewhat confusingly conflates a couple different missions which NOAA's organizational structure that breaks out very differently than how the Europeans do it. As a result, it is a mistake to compare ECMWF with the NWS. I'm oversimplifying dramatically here, but in the NOAA model, the National Weather Service provides daily weather prediction from their operational systems-- responsible for cranking out routine weather predictions several times a day on a rigid timescale. These models are not appropriate for hurricane tracking, but instead identifying systems that may need a closer look. Meanwhile, severe event prediction, such as hurricanes, rely primarily on external systems, at ESRL, GFDL, as well as outside of NOAA at the DoD, DoE and elsewhere. This burst capacity provides dozens of tracks that differ in their physics and complexity and are spread far and wide across the US government. The Europeans operate on a much different model, partly as a result of far more infrequent catastrophic weather events such as hurricanes.

For example, while the European prediction systems sit at #37 & 38 on the top 500 with 612 GFlops, the similarly-sized NOAA R&D system Gaia is at #40 with 566GF. The NOAA operations site simply isn't comparable.

As for the daily weather prediction, the differences in model resolution (15 vs 40km) has more to do with geography than with funding. Quite simply, the contiguous US is much simpler and more homogenous than Europe. Our west coast is a near featureless north - south line, followed by the N-W rocky mountains, a nice wide, flat great plains, and then a relatively simple N-S east coast. Compare that to the complex geography of Europe. We run somewhat lower resolution, because we can, and get somewhat similar reliability to what is necessary in Europe. You also need to distinguish for land-based versus oceanographic models-- which are run separately, and aimed at different products for different audiences.

One such advance is the ability to run many simulations of a model and generate probability distributions for forecasts. Weather is, to an important degree, a chaotic system. Run a model with the same initial conditions twice, and the results will rarely be exactly the same.

There are a few errors in this story.

Weather is a chaotic system, you are correct. But your following statement is wrong. Chaotic systems are deterministic: if you "run a model" with the same initial conditions twice, you will get exactly the same results, every time.

What characterizes a chaotic system is its high sensitivity on initial conditions. If you run two simulations with slightly different initial conditions, you get vastly different results. This should be compared to non-chaotic systems where slightly different initial conditions yield slightly different results (and sometimes, equal results). Weather is a chaotic system, therefore if you only know the current state of the climate system (temperatures, wind directions and speeds, etc) up to some uncertainty, your predictions become less and less accurate as you try to predict further into the future. You never know the current state exactly; this is why, as some have pointed out already, no meteorologist institute tries to make accurate predictions beyond 72 hours.

You also seem to have mixed up the "probabilistic vs. deterministic" dichotomy with the "chaotic vs. non-chaotic" one. Probabilistic models have their place in climate prediction, because they allow you to model the uncertainty I mentioned above. But it's important to understand that these are two separate characteristics of a system: it can be deterministic and chaotic (as the real weather is, unless you take into account quantum mechanics), probabilistic and chaotic (as our models of climate should be if CPU power allows), deterministic and non-chaotic (Newtonian mechanics, fluid mechanics, etc), or probabilistic and non-chaotic (can't think of an example which makes sense for weather prediction).

I would not be surprised to learn that anti-climate-science politicians have played a role here.

Actually, if anything, it is the opposite. Within NOAA, climate-change research is much "sexier" politically than simple weather reports. As a result, climate change oriented systems get far more earmarks and legislative attention than mundane, operational weather prediction. The recent NOAA NWS operations system upgrades have been marginally sized, while NOAA R&D, and in particular GFDL, have seen some very large system upgrades over the last few years, at both Oak Ridge and West Virginia.