A blog about weather, climate, and science. Occasionally, we'll post on other topics of interest.

Saturday, May 28, 2011

Selective Siren Activation, Part 3

Shawna and I drove through downtown Joplin minutes before the tornado hit. Sirens were going, but we could not see the tornado, and many people were out and about seemingly unaware or unconcerned. We had been following this storm complex for a couple hours without observing any tornadoes, and though we were a little nervous, we weren't anticipating anything of the magnitude that buried Joplin around 5:45 pm CDT.
Storm chaser and meteorologist Jon Davies, from his blog here.

Stan:

Guest, an eye-witness I spoke to said people at a driving range kept right on hitting golf balls even as the tornado sirens were blaring.
Stan Finger, Wichita Eagle web chat, Monday.

a CNN Wire report quoted Alexa Wattelet, in Joplin at the time the storm hit, as saying that "the sirens always go off, so no one thought anything of it."

Joplin resident Rick Morgan thought about it before taking cover.

“They go off, and it’s like, you know, tornado never comes, it seems like,” he told CNN on Monday.

Kansas City Star

What might cause this seemingly irrational behavior? I have a theory: We have sounded the sirens so often that people have come to ignore them. I've written two posts in the last six days on this topic. Go here and here to read them.

From 1957 to 2005, the National Weather Service issued tornado warnings by whole counties. This was appropriate because our skill at locating and tracking tornadoes was not very good. So, alerting relatively large areas made sense.

With the advent of Doppler radar in the 1990's and more accurate short term prediction tools, the National Weather Service switched to storm-based warnings, i.e., warning only the area(s) in the path of the storm. An illustration of the advantages is below:

Click to enlarge.

While visiting St. Louis earlier this week, I learned that they have a policy of sounding tornado sirens not only in all of St. Louis County when a warning is issued for any part of the county, but sounding the tornado sirens in all of St. Louis County when a tornado warning issued for an adjacent county!

In this hypothetical example, sirens would be sounding in Mehlville and Oakville
nearly 50 miles away from the tornado!

On Wednesday, a tornado warning was issued by the National Weather Service for a funnel cloud that followed the path indicated by the arrow. Photo is the actual funnel (located at the arrowhead) as viewed from our hotel (purple pin). The NWS issued an accurate warning for the path of the storm.

Path of the funnel cloud Wednesday with actual
photo of the funnel cloud we tracks from South St. Louis County

Yet, tornado sirens went off over all of St. Louis County. I've circled some of the cities in which the sirens were sounded (there were many more) that were not threatened at any time.

One would think this would prompt an examination of procedures since a false alarm occurred that affected more than 500,000 people. Unfortunately, it did not. KMOX radio interviewed me and interviewed St. Louis County officials. The story is here. Here is what the emergency managers had to say,

Defending the policy, the Acting Director of St. Louis County Emergency Management, Bill Roach, says the sirens mean one thing.

“Those sirens are an early warning device,” Roach said, “They don’t mean people should immediately run down in their basement and seek shelter. What they mean is you should seek additional information.”

Roach says the current policy saves lives, because tornados can change directions, and trying to guess where to selectively warn people could lead to deaths.

Trying to guess? This isn't 1964. Meteorologists know the direction of movement of these storms. Plus, the National Weather Service adds a margin of safety. There is absolutely no scenario where (to pick one example) Ballwin was at risk from that storm.

Saves lives? I believe this gross overwarning risks doing the opposite. When you sound the sirens time and time again in areas where there is no threat, people stop paying attention. I saw it in Lawrence and Overland Park Saturday and again in St. Louis on Wednesday.

It is long past time to rethink countywide and multi-county tornado siren activations. Sirens should only be activated in areas where there is a genuine threat.

19 comments:

This is what I've been saying all along. Ignoring the sirens because "they go off all the time, and nothing happens" is just stupid. The incentives will always be skewed in favor of warning people with marginal risk, because failure to warn someone who gets hit will always be seen as worse than warning those who won't. And there will always be people who don't take the warnings seriously.

Yes, it would be nice if public safety departments bought your polygon product, but it still won't change the fact that those polygons themselves are wide enough relative to the actual path of storm damage that most of the buildings inside the polygon will not be damaged. There will always be a lot more "false positives" than people who are actually affected. We must educate people to do exactly what Roach and I are saying, and turn on their local TV/radio broadcast to find out the details. The real problem is the people who hear the siren and don't even bother to get those details.

In the case of Joplin, we'll never know how many people died because they ignored the warning; it doesn't do a bit of good to take cover in your bathtub with a mattress on top of you when an EF-5 scours your entire house off the face of the Earth. There are many buildings that just don't have an emergency shelter able to survive a direct hit from one of these beasts.

"it doesn't do a bit of good to take cover in your bathtub with a mattress on top of you when an EF-5 scours your entire house off the face of the Earth."

I agree with most of your comments but the above is too general. Even a monster like Joplin does not produce F-5 winds over its entire path and taking shelter in a bathtub in the middle of the house is certainly better than nothing. Plus, in the 98% of the tornadoes that are not F-4 or -5 intensity that is excellent advice.

When EM's sound sirens 50 miles from the tornado they are doing the public a terrible disservice. It is no wonder people are ignoring sirens and -- possibly -- losing their lives as a result.

One area to explore on this issue (besides the technical/budget limitations of a selective siren system...I think you've addressed that several times) is the various fear local governments have of liability issues. I briefly worked in a 911 center and you would be amazed at how much of the training focuses on how to avoid making the county liable for wrongful death/injury lawsuits.

In fact, if someone called 911 during a tornado warning and asked what to do, we could not advise them to go to the basement, lest debris fall into the basement and injure or kill them. We had to use the more generic, totally unhelpful "seek shelter" and "take cover."

I suspect we have an entire generation of emergency managers throughout the country who have been trained with this philosophy in mind; saving lives is still the goal, but it must be done in a way that completely eliminates any possible claim of negligence. Hence, the tendency to overwarn.

Thanks for writing this up. I think that articles like this might help to explain why so many people have been killed by severe weather this season. The primary reasons are obvious, but something's definitely up with the way warnings are being received by the public.

I think three things are active here, first is risk aversion. It is much easier to sound all of the sirens than make a decision as to which sirens to sound. Sounding all of them means that the risk of making an error is 0. It turns the warnings into a brainless activity that has no other options to chose from.

Second is related to the first, liability. The direction of warnings in today's society is going in the opposite direction. You have warning for not drinking gasoline, eating wax fruit, and a whole host of others that boggle the mind. In this environment virtually no institution would want to assume that risk without some statutory protection. They would be un-insurable, leaving the community open to massive damages.

The third issue is change. "We have always sounded all the sirens no matter what, why should that change?" "You cannot predict where these storms will go". Unfortunately, this type of change requires that the person understand that the technology has changed, and yes we can now with good accuracy predict the path of the storm. Like many things in life it may take the retirement of these leaders to overcome that bias.

The Joplin tornado is a wake up call to the EM community. Despite what they as a group think, the public in general are ignoring warnings due to the high false alarm rate. Given the technology available today we should not be warning communities 30 to 50 mi away from a storm, and until we increase the resolution of the warning system more lives will be needlessly lost.

"Even a monster like Joplin does not produce F-5 winds over its entire path and taking shelter in a bathtub in the middle of the house is certainly better than nothing."

You completely missed my point, which is a neat trick. You had to clip out the first part of the sentence to quote it so horribly out of the context in which I stated it.

I'm simply saying that we don't know how many people of the people died in Joplin actually died because they had become desensitized to sirens. Some of the dead did the best they could (in the minutes after the warning was issued; I'm not talking about the decisions they made about where to live/work, re: the quality of emergency shelters), and their best wasn't good enough. That is simply a fact, which doesn't even appear to be arguable.

From that, you seem to be infer that I'm somehow suggesting people not even bother trying to save their lives. I'm not.

Since so many people are apparently dumbed down to the point that they can't understand anything more complicated than a 15-second sound bite (especially when someone takes it out of context), how's this for a soundbite:

Even if the warnings turn out to be "nothing"... five out of six times, that's the same odds as Russian Roulette. Do you still want to ignore the sirens?

Whenever I attempt to persuade a person to perform an activity, the first rule is always never work against human nature if you can avoid it. This applies to education, commerce, factory production and life in general. You cannot expect people to perform at a particular level if the stimulus works against the activity you wish to occur. False alarms be it with engine lights, your mother counting to 3 then 10 then 100, car alarms, or tornado waring sirens sounded for an storm 30 miles away will affect behavior. No denying that fact will make it not so. It is basic human psychology. To assume that everyone will react the same to a weather warning after having sat through 20 alarms that did not even get close to them personally is not reasonable.

If we take my county as an example over the past 6 years 36 warnings have been issued resulting in the sirens being sounded. in only one instance was my town even in the warned zone. Thus, the ratio is 36 to 1. If we talked actual tornadoes on the ground it would be 0. We must do better in the warning department, and technology should lead the way.

I would like to see a survey of the individuals involved in the Joplin incident to know how they reacted to the warnings. The anecdotal evidence so far points to many people ignoring the warnings. To have 19 minutes of warning, and still have people in their cars and shopping as if nothing was wrong with the world is a travesty. One that we should evaluate and attempt to improve upon.

Fine. Let's change the odds of Russian Roulette a bit. Our referee will take six identical six-shot revolvers, put one bullet in one of them, while assuring the other five are unloaded, and randomly arrange the pistols. That gives you 1/36 chance, same as ignoring the next warning.

Are you going to play?. What if we add eleven more guns, so the odds of killing yourself are less than 1%? Would you play at those odds? Let's make it a thousand to one? You ready to play yet? Or are you going to grab that mattress and get in the bathtub, even if the odds of getting killed if you don't do it are reduced by less than one in a thousand?

If the authorities only warn those people who have a better than 50% chance of being hit, then most of the people who do get hit won't have been warned. I know this sounds counterintuitive, but even with the current state of meteorology, we can't nail down a tornado track that well. Let's try an example where life and death aren't on the line to see how it's possible for the most likely outcome to be an unlikely event:

If Super Bowls were tornadoes, we might issue a "watch" for any team with a winning record at mid-season, and a "warning" for those that make the playoffs. Each team would have a 1/12 chance if the probabilities were evenly distributed; on the other hand, if the teams with a first-week bye were 3x as likely to win as the wild card teams, and the other division winners were 2x as likely, those wild-card teams would each have a 1/24 chance of winning, the division winners that host the first-round games would each have that "average" 1/12 shot, and the teams with first-round byes would each have 1/8 chance. There's over 88% chance of a false positive.

So suppose we decide not to issue a "warning" for wild-card teams. Over the 42-year history of the Super Bowl being open to teams that didn't win their division, seven have won ('69 Chiefs, '80 Raiders, '97 Broncos, '00 Ravens, '05 Steelers, '07 Giants, '10 Packers). That gives us a false negative rate exactly equal to Russian Roulette with a six-shooter, (also what the 3:2:1 probabilities above would predict) and we reduce the false positive to just over 74%. Even if the most likely outcome occurs, 7 teams will have been "warned" that don't win.

Probability and statistics are not well understood by the average person. (If they did, Las Vegas would be a small town.) There simply is no way to use a binary "siren is/not on" to convey adequate information about the risks involved. People must be taught that the siren indicates sufficient threat level to seek out that additional information.

Maybe the sirens could also blast out audible information ("Radar indicates a tornado with debris cloud approaching this area. Take cover immediately" vs. "A tornado was detected approaching Gardner, so don't drive that direction.") But then La Raza would complain because the warnings aren't in Spanish, and you know that Standard English is really White English, so you should have it in African American Vernacular English too, and qatlh ta' chaH ghobe' jatlh tlhIngan?!?

Monster, first I did not state that the probability of having an effective warning was one in 36. I was stating that the ratio warnings issued to having those independent events having my house in the NWS warning cone was 1 in 36. This is not a probability, it is an error rate for a particular warning method. If another method were used then that rate would be reduced according to the resolution of the system. If the mesh is large (coarse) then the rate may drop to 1 in 3, if it is fine then 1 out of 1. 1 in 3 is much better than 1 in 36. However this is still not a probability, just an error rate. To convert this into a probability, you would have to understand the geographic coverage of the mesh, the distribution of the warnings, and a host of other factors. You cannot just take a ratio and assume that it is a normal distribution, and at no point did I suggest so.

As far as the analogies you place here, they are Non Sequitur in nature. As stated above the warning error rate was not a probability. Each storm was tracked and warning cones shown, with only one described to to have been a threat to my home by location. A better analogy would have been to attempt Russian Roulette without any weapons, since the probability was essentially 0 for the 35 non-threatening instances that were warned. I do not need to be warned about a tornado in Kansas if I live in Iowa. Why should I be warned about a storm 30 miles away? If we took your reasoning to its end point every tornado warning would sound all across the US. What Mike is saying and I agree with him is let's stop doing the same thing especially since in doing so it will reduce false alarms.

The second part of your statement about the 50% issue etc. is not valid as well. With the advances in RADAR available to the NWS and other organizations they can identify where these storms are located, and predict the movement very accurately. If you look at the warning cones and overlay them on the damage paths of the tornadoes the storm generate, they are very accurate. If you are outside the cone the probability of being struck by the tornado is basically 0. Thus, your arguments are vacant in the warned areas, since with the guard bands given to the cones make it moot.

As far as tornadoes being like the Superbowl, that arrangement is vacuous as well. Each tornadic incident is an independent occurrence, meaning that a previous occurrence of a tornado in an area will not affect an event days or years later. Your team analogy does not follow because previous outcomes affect future events. First, not every team is as likely to win against another. Second, since the team’s performance earlier in the year affects their standing later the tests are not independent, especially when you conciser that injuries play a role in winning as well. Thus, this argument fails to support your point.

Probability is not something that a person should need to understand in a warning. Either the storm threatens them in that area or not. Given the state of technology today that level of resolution is available. You do not get warned about an isolated westbound construction project while traveling eastbound on the interstate highway, it is irrelevant information. Your argument really suffers with the last paragraph, making whatever you say ignorable just by tone and bias.

Weather warnings need to focused and effective for them to be taken seriously. Having a siren go off to alert someone of a problem 30 miles away is a waste of time and reliability capital. If someone hears a siren and knows that the siren is there to warn them of an immediate danger that requires action on their part then they will follow it. On the other hand if they are warned for every incident even those that do not affect them, then warning fatigue sets in and you get what happened to some of the people in Joplin. They have heard the sirens go off so many times for stuff that does not affect them that they now ignore it. As I stated above we should be working with human nature not against it.

"If you look at the warning cones and overlay them on the damage paths of the tornadoes the storm generate, they are very accurate. If you are outside the cone [sic] the probability of being struck by the tornado is basically 0"

But "basically 0" is not "0". The choice of the exact point of "basically 0" is what we're basically arguing about here.

"Either the storm threatens them in that area or not." Probability is not binary. We're asking public officials to draw a line. The incentives are always in favor of minimizing false negatives at the expense of having more false positives. If the odds are one in a thousand that the cell approaching Gardner will hit Westwood in a half hour, the Johnson County public service people will always trigger the zone the Westwood siren is in. Every time. That is human nature every bit as much as the "Crying Wolf" phenomenon.

"They have heard the sirens go off so many times for stuff that does not affect them that they now ignore it."

Despite the fact that these warning polygons are "very accurate" compared to the warnings when I was a kid, these polygons are still large enough that for those inside them the probability of being "affected" is significantly less than the probability of not being hit. Even if we only warn the people you and Mike want us to warn, they will be able to react to the (somewhat reduced rate of) false positives exactly as they are reacting to them now.

"Each tornadic incident is an independent occurrence, meaning that a previous occurrence of a tornado in an area will not affect an event days or years later."

You have seized upon the irrelevant differences between predicting Super Bowls and tornadoes, as if it affects what they have in common: The most likely outcome, as determined by the best forecasters in the field, is still unlikely to occur precisely as forecast. The best team in the NFL is still more likely not to win the next Super Bowl than to win, and the average person in a storm track polygon more likely not to be hit by the tornado than to be hit.

Do bear in mind that we do NOT want a warning limited to those people with a greater chance of being hit than not hit. That would be a horrible disservice to the public. This is why people have to be educated to understand that most of the people in a polygon won't be hit, but that there is a chance that any individual will be, just like most of the people who take the 1/6 odds of Russian Roulette won't die. I keep coming back to that as an example that most people would understand of how something can be unlikely, and yet still worth protecting against.

"Your argument really suffers with the last paragraph, making whatever you say ignorable just by tone and bias."

I guess "bias" means that I think the self-appointed defenders of ethnic minorities are capable of making ridiculous demands, and "tone" means I don't pretend to respect those demands. Fortunately, I'm not a politician, so I can speak the truth. Treating these demands as anything less than laughable just encourages even more bizarre demands.

I was trying to convey the futility of trying to come up with a solution, given how these demands are pressed against anyone who dares do anything that can somehow be found "unfair" to some group or other. Each of the examples has some referent in reality. The ultimate absurdity of someone insisting on a Klingon translation comes from a news story I saw a few years ago about an Oregon county office needing a Klingon translator, which apparently was rescinded after AP picked up on it. I am insufficiently creative to make up a story like that.

"I learned that they have a policy of sounding tornado sirens not only in all of St. Louis County when a warning is issued for any part of the county, but sounding the tornado sirens in all of St. Louis County when a tornado warning issued for an adjacent county!"

That is not the policy of St Louis County.The policy is to sound the sirens when there is a warning issued for St Louis County and to sound the sirens an additional time if a trained weather spotter spots a funnel cloud that is in St Louis County or an adjacent county with a path that takes the siren into St Louis County while St Louis County is under a Tornado warning.

"Yet, tornado sirens went off over all of St. Louis County. I've circled some of the cities in which the sirens were sounded (there were many more) that were not threatened at any time."You circled Eureka, Ballwin, Chesterfield, Kirkwood, Bridgeton, Hazelwood, and Florissant.

Hazelwood and Florissant were both in the path of the funnel cloud spotted over the Galleria,as well as Berkeley, Kinlock, Jennings, Dellwood, Bellfontaine Neighbors, Riverview, and Blackjack, according to the archived tornado warning, all of which are well out of the path of the arrow you drew.There was actually significant damage in the Jamestown Mall area.The second second funnel cloud spawned by the cell passed over Kirkwood, as well as Arnold, Fenton, Valley Park, Sunset Hills, Crestwood, Lakeshire, Town and Country and Webster Groves, again, according to the warning issued by the national weather service.And finally, the third warning, the one you are specifically referring to (though all happened in a matter of less than a half hour), went over Bridgeton, Hazelwood, and Florissant, all cities you circled, as well as Richmond Heights, Olivette, Clayton, U City, Overland, St Ann, St John, Woodson Terrace, Bel-Ridge, Normandy, Berkeley, Cool Valley, Kinloch, Ferguson, Dellwood, Black Jack, and Glendale.All of these communities were in the NWS warning issued for the very funnel cloud you saw.All of those communities are well outside of the path you drew.

Here is the more difficult issue though. Take St Louis County as an example. They are using Whelen WPS-2010s for their sirens. Those have a 70 decibel range of 6100 feet. So, over 2 miles across. Their 50 decibel range, still easily loud enough to be heard outside, is 24,400 feet. That means that a single siren can be heard across an area over 9 miles wide. (I can clearly hear sirens in St Charles County from my house, even though St Charles County is 6 miles away and uses much smaller sirens.)

Assuming you activate only some of the sirens in a county, which ones do you activate?

Do you activate those sirens which are physically in the warning area? There will likely be households in the warning area that will not have a covering siren activate.

Do you activate sirens whose 70 decibel range intersect the warning area? There will be households up to 2.3 miles outside the warning area that will be covered by those sirens. Meanwhile, houses up to 5 miles away will hear the sirens, even though the sirens closest to them will not activate, causing a phantom siren effect where they think they hear a warning, but are unsure.

Do you active those sirens whose 50 decibel range intersects the warning area, ensuring no phantom siren effect at all? At that point, any warning more than 9 miles wide will activate every siren in the county anyway no matter where it passes.

1. "That is not the policy of St Louis County.The policy is to sound the sirens when there is a warning issued for St Louis County and to sound the sirens an additional time if a trained weather spotter spots a funnel cloud that is in St Louis County or an adjacent county with a path that takes the siren into St Louis County while St Louis County is under a Tornado warning."

That may be true but that is not what St. Louis Co. emergency management was telling the media Friday.

2. I don't understand your comment. I was referring to the later warning. The cities I circled were clearly outside the path of the tornado that passed just east of downtown St. Louis.

You are correct that SOME of those cities were in the path of the earlier warning. However, downtown STL was not yet the sirens were going off there. They were also going off in far west county behind and far to the west of the circulation's path.

3. That is why the sirens in the polygon should be sounded with, perhaps, a one mile buffer around the polygon (to pick up sirens that are physically outside the polygon but can be heard inside the polygon). While there may be some "slop" (i.e., sirens heard outside the polygon) that would be far superior than sounding the sirens in Eureka when the threat is in downtown St. Louis.

If you want to pick up every siren that can be heard inside the polygon, you would need to go out to at least a 12200' foot buffer, probably a 24400' buffer if you want to cover sirens that would be audible outside only. This is using the 6100' 70 db range of the WPS-2910 and 50 db as an audible noise level.

Since those same sirens can be heard 24400' in the other direction too, though at much reduced loudness, you would have effectively a 48800' (9.25 mile) buffer of sounding sirens around the warning area, with the decibel level lower the farther away you were from the actual warning polygon, but still cleanly audible.

While this would be effective in covering the warning area, St Louis County is only 19 miles across at its widest N/S or NW/SE diagonal (few, if any tornadoes travel across the county south to north). You will be activating at least half the county for all but the narrowest slivers of warnings, and the entire county for nearly every warning.If you throw that kind of buffer around the warning polygon for just the last funnel cloud's warning from last week, you get every single siren in the county (missing only a narrow swath out by Howell Island/Babler Park which does not currently have sirens).

The other warning active at the same time as the Lemay warning (the one that sent a funnel cloud over the Galleria) would have covered the entire county with that buffer.

Incidentally, downtown St Louis is not part of St Louis County's siren system. They are a completely different system.

You are correct that there were areas to the west that received warning that would not have received a warning with a sector based warning (depending on how you draw the sector buffer of course), but based on the cities in the warning that were not in the eventual path of the funnel cloud, even with a sector based warning, the false alarm that would have occurred would have "affected more than 500,000 people."

Much of the area not covered by the third warning were covered by the second warning that was occurring almost simultaneously over Kirkwood, Arnold, Fenton, Valley Park, Sunset Hills, Crestwood, Lakeshire, Town and Country and Webster Groves. No funnel cloud was spotted with that warning, but another large swatch of the county was still in range of it.

If St Louis County were to switched to a sector based warning, how many less warnings would the average person hear over the course of a decade? How many times would people in the path of a tornado not receiving warning who would have otherwise received warning with a countywide alert? (e.g. the areas west of I-270 in the Good Friday tornado?)

I work for the NWS, and I have recently read a blog by James Spann about NWS trying to warn on brief spin-up tornadoes within squall lines.

I am personally worried that we are doing a disservice and watering down Tornado Warnings when we do this. If you think there will be widespread damaging winds and have a Severe Thunderstorm warning out, the impact will be greater for more people than a tornado.

However, if you issue Severe Thunderstorm warnings and then try to put out tornado warnings to cover a brief weak tornado spin-up, then you are watering down the effect of a tornado warning for an event like Joplin MO or the April 27 2011 event in AL/MS/TN etc.

Not all tornado warnings are created equal. The Tornado Emergency has some merit, but we don't always have the skill to know when the big one is actually going to happen.

The Moderate Risk day in Joplin was not an obvious day for a record killer EF5 tornado. Sure, the potential is higher for a Joplin like tornado on Moderate or High Risk days, but sometimes Mother Nature overachieves and sometimes we just get unlucky about where the big tornado will strike.

The comments about sirens going off all the time and nothing happens is partially a problem that the NWS has created. But in order for us to back off on how many tornado warnings we issue, we need to feel we won't get raked over the coals for missing every little weak tornado that has far less potential to kill people or create massive damage.