22 October 2012

Mischaracterizations of the L'Aquila Lawsuit Verdict

Today in Italy 6 scientists were convicted of manslaughter in an Italian court based on their provision of allegedly faulty information. Nature reports:

At the end of a 13-month trial, six scientists and one government official have been found guilty of manslaughter and sentenced to six years in prison. The verdict was based on how they assessed and communicated risk before the earthquake that hit the city of L'Aquila on 6 April 2009, killing 309 people.

There is a popular misconception in circulation that the guilty verdict was based on the scientists' failure to accurately forecast the devastating earthquake.

Defying assertions that earthquakes cannot be predicted, an Italian court convicted seven scientists and experts of manslaughter Monday for failing to adequately warn residents before a temblor struck central Italy in 2009 and killed more than 300 people.

The court in L'Aquila also sentenced the defendants to six years each in prison. All are members of the national Great Risks Commission, and several are prominent scientists or geological and disaster experts.

Scientists had decried the trial as ridiculous, contending that science has no reliable way of predicting earthquakes. So news of the verdict shook the tightknit community of earthquake experts worldwide.

"We are deeply concerned. It's not just seismology which has been put on trial but all science," Charlotte Krawczyk, president of the seismology division at the European Geosciences Union (EGU), told AFP.

The verdict struck at scientists' right to speak honestly and independently, she said in a phone interview from Germany.

"All scientists are really shocked by this," said Krawczyk. "We are trying to organise ourselves and come up with a strong statement that could help so that the scientists do not have to go to jail.

"People are asking, 'Is this really true?' 'What does it mean for us?' And, 'What does it mean for talking in public about risks?'"

"People are stunned," said Mike Bickle, a professor of earth sciences at the University of Cambridge.

Roger Musson at the British Geological Survey (BGS) said the verdict was "unbelievable". He and other seismologists said it was impossible to forecast an earthquake, and scientists pressed to give a black-or-white answer could unleash panic or lose all credibility if nothing happened.

Unfortunately, such characterizations of the he lawsuit are simply wrong: the scientists were not on trial for their failure to predict the earthquake.

The trial in L'Aquila has drawn huge international attention, as well as outrage and protests. In 2010, more than 4000 scientists from Italy and around the world signed an open letter to Italian President Giorgio Napolitano, calling the allegations “unfounded,” because there was no way the commission could reliably have predicted an earthquake. Alan Leshner, CEO of AAAS (the publisher of Science) called the indictments “unfair and naïve” in a 2010 letter to Napolitano.

Yet as the trial unfolded here over the past year, a more complex picture has emerged. Prosecutors didn't charge commission members with failing to predict the earthquake but with conducting a hasty, superficial risk assessment and presenting incomplete, falsely reassuring findings to the public. They have argued in court that the many tremors that L'Aquila experienced in the preceding months did provide at least some clues about a heightened risk.

Meanwhile, a recorded telephone conversation made public halfway through the trial has suggested that the commission was convened with the explicit goal of reassuring the public and raised the question of whether the scientists were used—or allowed themselves to be used—to bring calm to a jittery town.

I discussed some of the dynamics at play in my Bridges column of October, 2011 (here in PDF):

On March 31, 2009, in L’Aquila, six days before a deadly magnitude 6.3 earthquake killed 308 people, Bernardo De Bernardinis, then deputy chief of Italy’s Civil Protection Department , and six scientists who were members of a scientific advisory body to the Department (the Major Risks Committee) participated in an official meeting and press conference in response to public concerns about short-term earthquake risks. The public concerns were the result of at least two factors: One was the recent occurrence of a number of small earthquakes. A second factor was the prediction of a pending large earthquake issued by Gioacchino Giuliani, who was not a seismologist and worked as a technician at Italy’s National Institute of Nuclear Physics.

The deputy chief and scientists held a short one-hour meeting and then a press conference, during which they downplayed the possibility of an earthquake. For instance, De Bernardinis went so far as to claim that the recent tremors actually reduced earthquake risks: "[T]he scientific community continues to confirm to me that in fact it is a favourable situation, that is to say a continuous discharge of energy." When asked directly by the media if the public should sit back and enjoy a glass of wine rather than worry about earthquakes, De Bernardinis acted as sommelier: "Absolutely, absolutely a Montepulciano doc. This seems important." . . .

. . . in L’Aquila, the government and its scientists seemed to be sending a different message to the public than the one that was received. Media reports of the Major Risk Committee meeting and the subsequent press conference seem to focus on countering the views offered by Mr. Giuliani, whom they viewed as unscientific and had been battling in preceding months. Thus, one interpretation of the Major Risks Committee’s statements is that they were not specifically about earthquakes at all, but instead were about which individuals the public should view as legitimate and authoritative and which they should not.

If officials were expressing a view about authority rather than a careful assessment of actual earthquake risks, this would help to explain their sloppy treatment of uncertainties.

The case is likely to be appealed, so the current verdict is not the last word. While the verdict rests on finer points of Italian law and jurisprudence, the issues at play are not accurately characterized as a failure to accurately predict an earthquake, or even more broadly as science vs. anti-science. The public responsibilities of government officials and the scientists that they depend upon are too important to characterize in such cartoonish fashion.

23 comments:

So what does this say about your confidence in the competence of the mainstream media? Or does this massive failure to get the facts straight point more to bias than to incompetence? Which is a better defense - too stupid to know better, or too biased to be held back by the truth?

Thanks for this. I was also frustrated with the misrepresentation. On the other hand, I've also heard the "prediction" issue in quotes attributed to the defence lawyers, so it makes me wonder if they also argued from this position. If so, it seems like a mistake. (obviously not trying to propagate misinfo myself, but I was struck by this aspect in the reporting)

Roger, I admire the delicacy and precision of your reporting here. Thanks.

I may not be so delicate because I have an emotional bias against the tide of projections of science certainty being trumped above normal human discourse ;)

The fact that the underlying complexity here seems a handy excuse for laziness and bias in the reporting of this legal case seems ironic. And doubly so because many self-describe defenders of "science" appear too willing to allow this "simplification".

I don't doubt there may be future revisions of the details, based on the emotions and location of the trial, but the depiction as "Scientists imprisoned for not predicting rare event" is such a clear misinterpretation - and can be seen even when rummaging through the body of the headlined stories themselves - that the promulgators of this meme must extend all the way through those people with a casual ignorance to those with a wilful dislike of an honest interpretation.

When it is clear that the legal complexities that *can* possibly be understood by lay people, but apparently can’t be trusted to be handed to them, then you see the same human nature at work, that is so often supposedly transcended by scientific purity but just hidden under its cover, put to work for a self-serving purpose.

"Whiplash would describe my reaction to this post. It's disorienting to sit through the debate between Pres. Obama and Gov. Romney tonight, where the truth, as usual, had to wait for disambiguation by the MSM, as though they could be trusted to do that job, and then to catch this correction to the outrageous Italian earthquake story.

I just pointed the Times reporter to this link. Perhaps at least they will get a start on cleaning up after themselves. I will really start to worry if the story goes no further than what is out there now."

While I understand the distinction you are making, I can't see that it affects the egregious injustice of the case. These scientists were called upon to make a risk assessment based on their expertise, which they did. It was wrong. Shit happens.

If they had been right, no one would have cared whether they exercised enough diligence. No doubt they believed they had. Earthquakes are still inherently unpredictable from our point of view, more diligence would not have affected that, or necessarily have led to a different answer.

And the next time a group of scientists is asked for an assessment of earthquake risks (or hurricane risks, or tornado risks, or anything inherently difficult to predict), they will either a) cover their arses by predicting disaster regardless of their own judgement, or b) get the hell out of the business before they end up in jail.

I knew from the start that this trial was not about the scientists failing to predict an earthquake. I was totally aware of the issues involved. I have better sources than the news agencies you have stated above. That is not the issue.

The issue is 6 years in jail for making a mistake. Yes, maybe bordering on professional negligence by not assessing the risk carefully enough. But let's not forget it was a mistake. Not a deliberate criminal act. A mistake.

The judge has not taken into account the entire careers of these scientists. They may have potentially contributed to the saving of millions of lives through their research.

The judge, on behalf of society, is essentially saying to the scientific community "we are not grateful". Would he rather that scientists never work on problems for risk that they may be wrong and people might die? We would surely still be living in the middle ages if that were the case. Scientists wouldn't have been able to develop vaccinations, eradicate Smallpox from the world, X-rays, chemotherapy...the list is endless.

If the appeals court find that the judge was wrong in this case (which I sincerely hope happens), should the judge be sent to jail for six years? Why not? That is the precedent he has set for those that make mistakes. He should himself be bound to the same standards he places on others.

"The public responsibilities of government officials and the scientists that they depend upon is too important to characterize in such cartoonish fashion."

This is still no basis for trying and convicting scientists involved in risk assessment. The scientists they "depend on"? That means little given the poor state of earthquake prognosticating. Sure, the scientists involved could have been more quantitative and said there is a higher probability of an earthquake occurring in the near future, where "higher" means 1% or less. Would many people have paid attention to such a low probability prediction? Would officials have used it as a basis for carrying out actual evacuations? And how many false-alarm evacuations (and there would be many such false alarms) would it take before people stopped paying attention to them or took government officials to court?

The real story is that Italian elected officials did not devote resources to improve building standards in this earthquake-prone region that would have saved lives over the long term given the certainty of an earthquake at some point and poor existing construction. Instead, its justice system decided to take on a scientific risk-assessment body as the major culprit. This is scapegoating of the worst kind and that's what's being called out in the news coverage of this tragic event.

There is another factor here. The racism of the British and American media in reporting the justice systems of non Anglo Saxon countries.

The McCann family were spirited out of Portugal by the British government. The Guardian concluded they were innocent because the Portuguese are utterly corrupt and incredibly stupid. The McCanns measured 9.5 on the OJ Simpson scale.

A 30 second investigation by myself discovered Dr McCann's membership of a British government committee on the safety of the nuclear industry. A fact never revealed in the British media. Yes, he was a scientist (and a doctor).

The Italian scientists have been given honorary white status because they are our (rational) heroes.

There was also the case of American Foxy Knoxy in Italy. The Guardian had to be careful because she absolutely didn't murder her British room mate in a sex game even if those dumb Italians believed she did.

Both those cases involved the suspect use of DNA evidence the Guardian believes can only be properly assessed by Anglo Saxons. In both cases, the assessment changed. Science Pinochios saved the day.

Scientists routinely claim knowledge, skill, and certainty which exceed reasonable understanding. They routinely conflate matters of philosophy and science, thereby sabotaging the legitimacy of science. They infer the origin of the universe and species from limited, circumstantial evidence and emergent patterns. They have on more than one occasion exploited esoteric physical phenomenon to rationalize and justify involuntary redistribution schemes.

Surely earthquakes, which exhibit a chaotic behavior (i.e. bounded but otherwise unpredictable), as global thermodynamics, should be equally predictable, and therefore it is it reasonable to expect an equally exacting accuracy from a consensus position.

Perhaps the missing link is an IPCC-equivalent organization, and an enthusiastic corp of journalists, to force a consensus on earthquake forecasting.

Anyway, the fallout from this incident was predictable. Much more so than the behavior of an incompletely characterized and unwieldy open system. The only question is who should be held to greater account, the scientists, the politicians, or other interests who have previously exploited and perverted science for personal gain.

And the next time a group of scientists is asked for an assessment of earthquake risks (or hurricane risks, or tornado risks, or anything inherently difficult to predict), they will either a) cover their arses by predicting disaster regardless of their own judgement, or b) get the hell out of the business before they end up in jail.

And yet when the South Island of New Zealand had a big quake, the scientists involved in assessment did neither of these things. They correctly observed that another series of small quakes was likely to follow, and that another big one was not precluded. That big one then struck months later, killing some in Christchurch.

The assessment of the Italian scientists in this case that small quakes were not an indication of a larger one to follow does not smack of science. It smacks of trying to quieten down an opposing view from a non-accredited source. That is, their public statements appear to have been non-scientific.

If they had been based on science, they would probably not be in the situation that they are in now.

It's not, and never has been, about the ability to predict. It's about telling the truth when people's lives are at stake.

What we have here is scientists issuing a statement calculated to induce the correct behaviour (don't panic) rather than provide accurate information, because they know that an accurate statement (there could be an earthquake) would lead to the wrong behaviour (panic and evacuation). There's no solution to this -- it would be irresponsible to issue an accurate statement that could lead to mass panic.

@12. drplokta said..."What we have here is scientists issuing a statement calculated to induce the correct behaviour (don't panic) rather than provide accurate information, because they know that an accurate statement (there could be an earthquake) would lead to the wrong behaviour (panic and evacuation)."

I suggest that scientists shouldn't be spending time trying to calculate ways to induce "correct" behaviour. Hypothetically if we accept they knew that the accurate information was that there "could be an earthquake" then they should have just said that along with any modifying statement of certainty.

"There's no solution to this -- it would be irresponsible to issue an accurate statement that could lead to mass panic."

I suggest the "no solution" feeling only arises when you just realise you might be responsible for something outside of your skill set - assuming that skills in seismology aren’t assumed to include skills in predicting mass human behaviour ;)

Leave those worries behind when you man up to whatever responsibility you have signed up for. I suggest if you have signed up to be part of an organisation called “National Commission for Forecasting and Preventing Major Risks” then you may know what is expected of you?

I don't doubt there could be eventual success in appeals on specific grounds of lack of culpability or scapegoating, but I am constantly bemused and disturbed by the fact many defenders rather choose to defend the scientists by an appeal to heightened indignation at the status of "scientist" being threatened.

This suggests to me that these supporters have a favour for a consequence free world for any scientist involved in policy - merely earned by their status as scientist – transferring away any need for human culpability – like erm priests?

There's no solution to this -- it would be irresponsible to issue an accurate statement that could lead to mass panic.

This is risible.

Scientists routinely warn that the East Coast of the US could have a major quake. They routinely warn that Naples lies under a major volcano risk.

Any signs of panic seen? No. Naples and LA continue to flourish.

Quite contrary to what you say, people blatantly refuse to take warnings seriously. There wouldn't even have been an exodus, let alone a "panic", if the scientists in central Italy had warned that a big quake was likely.

The area has been rocked by quakes all through human history. That a quake was going to strike the area some time was almost a certainty – although obviously the exact place and timing was uncertain. The locals know that, which is why they were questioning the "experts" in the first place.

Scientific experts are embedded in society and part of cultural elites. In this role they do not want to "rock the boat". Often they think they know well what is digestible by their political masters or by the general public. This is the structural problem of communicating science properly. Sometimes it takes courage to do just that. Court decisions and criminal justice will not improve this situation.

Excellent observation and something I became very aware of on this forum. I realised that my links to the Washington Post on the fossil fuel industry's support for AGW were about as useful as sponge bullets to academics because it doesn't fit the mainstream narrative.

I did it because it is useful to know the truth even if you can't repeat it. My original source was Prof. David F. Noble, a 'rogue' academic from Toronto who apparently doesn't like the internet. Others obviously post his material for him

I couldn't understand your defense of the verdict. I don't understand what's wrong about reassuring the public that the risks are low if the scientific analysis determines that the risks are low - and there doesn't exist any convincing evidence even today that the risk in similar situations is high i.e. that the earthquake was something else than really bad luck.

I think that reassuring the public that behaves irrationally is a good thing. But even if you disagreed, it's still very far from having an argument that the reassuring is a crime. A crime is a very different level.

Also, Mr Giuliani uses a method (based on radon gas release etc.) that isn't believed to work by the seismologists, and I don't really believe it myself. Don't they have the right to believe that this is not the right diagnostics? Is it criminal to think that Mr Giuliani is a hack? The fact that he made one right guess surely doesn't prove his methodology right, does it?

Moreover, even if you decided that the radon gas method is right, it's clearly unable to predict earthquakes 100% reliably, either. So in other cases, the radon gas people could reassure the public while people with other methods could warn and scream - and the latter could still be wrong in some cases. It's guaranteed by statistics that such things happen. So even if one accepts the problematic radon gas method, it isn't a recipe to avoid sentences against innocent scientists who just couldn't mitigate what is being blamed upon them.

The sentence is outrageous and while I think that your writing usually makes sense, this particular text of yours is incoherent.

what's outrageous about the verdict is that the scientists were sentenced for producing a prediction of a hardly predictable question – thing that simply can't be predicted reliably.

It is convicting totally innocent people as scapegoats - because they are claimed to possess a crystal ball that could tell them that "this time the situation was dangerous". There is absolutely no evidence that there was something different that would imply that the risk of a major earthquake was higher than in thousands of similar situations in which no large earthquake occurred.

The judge is trying to obscure this basic fact - that the scientists are being harassed for presenting their scientific conclusions and for being connected with a sad and unlikely event they could neither cause nor predict - and you are trying to obscure this basic fact, too. That's why you are saying the same incoherent rubbish as the judge and that's why you are defending the outrageous verdict.

I can speak (type, actually) to this from the angle of tornado prediction.

"drplotka" typed: "There's no solution to this -- it would be irresponsible to issue an accurate statement that could lead to mass panic."

Based on what evidence? This is unadulterated speculation, an archaic train of thought that has been debunked resoundingly in my realm of predictive science.

Five decades ago, explicit usage of the word "tornado" in U.S. forecasts was forbidden because it was thought to lead to "panic"--a sentiment similarly based on nothing more than idle conjecture.

We have made great strides over the last 30+ years in multi-day predictions of areas where the atmospheric environment is favorable for tornadoes--especially the swarms of violent, long-tracked specimens most responsible for death and damage. Lesser tornado events necessarily narrow down the lead time because they usually are dependent on more subtle and smaller-scale factors (such as outflow boundaries or variations in strength of capping) that may not be apparent until hours ahead, or even after storms start to form.

Yes, we are far ahead of earthquake science in prediction of the hazard. Nonetheless, explicit prediction of tornado risk on a daily basis since then has failed to cause "panic"--including the short-fused tornado warnings that urge immediate sheltering. Sure, some *individuals* panic in tornadic situations, usually after a warning is issued and when face-to-face with one that is bearing down, but this hasn't applied to mass groups. If anything, as Mark alluded, post-disaster surveys have shown that watches and warnings are not taken seriously enough, due to a lingering high false-alarm rate and assorted personal confirmation biases.

In the U.S., I would not worry about being held criminally responsible for a missed (or very low-probability) prediction unless one is shown to be grossly and purposefully negligent somehow.