Preferred method of conveying climate risk doesn’t work

The IPCC should forget readability and put actual numbers in its reports.

If I told you that something was "very likely," what would you consider the odds of it happening?

The answer isn't just academic. The Intergovernmental Panel on Climate Change (IPCC) reports on climate change use terms like "unlikely" and "virtually certain" to describe very specific degrees of certainty or probabilities of future events—information that informs government policy. And the challenges of conveying that information extend well beyond the climate. Pretty much any risk evaluation, from health to nuclear safety, involves some degree of conveying probability.

The IPCC's approach is to go for readability, using the phrases noted above instead of numerical values; it typically provides the translation between words and numbers in a table near the top of their reports. For example, crack open an IPCC report and you'll see that "very likely" means greater than 90 percent.

The problem is that these words don't convey enough information. Researchers conducted a survey of citizens in 24 countries, using 17 different languages. They found that regardless of the country or language, the replacement of actual numbers with words severely reduced people's ability to figure out probabilities. Remarkably, however, providing the numbers as well didn't help as much as you might think it would.

The survey focused on four terms used by the IPCC reports: "very likely," "very unlikely," "likely," and "unlikely." These words are meant to convey certainty within 10 percent for or against, and certainty within 33 percent for or against. The terms were either presented to the participants on their own or accompanied by the numerical meaning—so either "very likely" or "very likely (>90 percent). After seeing the phrase, the participants were asked what numerical probability the phrase was intended to convey.

One thing that was very obvious is that even if the probabilities are spelled out, people still have trouble coming to grips with them. A few people can apparently look at the phrase "very unlikely" and decide that it's meant to convey an 80-percent probability of something happening. In the same way, a few people felt that "very likely" was consistent with less than even odds. Whether this is a reading comprehension issue or simply surliness isn't clear.

But it is clear that having the numbers present tended to help people digest the probabilities. For each of the phrases, the average of the probability estimates provided by the participants were closer to the intended values when the numbers were present. This was especially true for the more extreme cases—"very likely" and "very unlikely." In these cases, the participants tended to undervalue the "very" aspect and suggest the intended meaning is much closer to even odds than it actually is.

These effects were consistent in every language and every population surveyed, suggesting that it's a general issue with human perception rather than something specific to local nuances.

The results show that the IPCC's approach, which may increase the readability of its reports, actually cuts down on the understanding of the reports' contents. And it does so in a way that has the strongest effect in those cases that either should significantly allay concerns or call for significant action to mitigate or prepare for high-probability risks.