All numbers, big and small, need to be put in context to be meaningful: a day that is 1˚C warmer than the previous one is hardly different but a 1˚C rise in the average temperature of the Earth would distort the whole ecosystem. A disease with a 0.1% mortality rate doesn’t sound too worrying until you consider that it could still cause the deaths of thousands of people every year if it is a common disease.

Dr Oliver Ratmann

Postdoctoral research associate, Duke University

Did you Know?

Statisticians use standard levels of ‘unlikely’. Commonly they use significant at the 5% level (sometimes written as p=0.05). In this case a difference is said to be ‘significant’ because it has a less than 1 in 20 probability of occurring if all that is going on is chance.

Background

Statistics are used to measure and make sense of the world. They are produced by the Government, political parties, the civil service, the Bank of England, opinion polls, campaign groups, social research, scientific papers, newspapers and more. But when confronted with stories such as “Crime rate rising again”, “Polls put Tories up to 7% ahead”, “Child heart surgery halted at hospital after four deaths” or “Swine flu ‘could kill up to 120m’”, how can we work out whether to believe them and what they really mean? This guide is not a lesson in statistics. It provides the questions to ask and identifies the pitfalls to avoid to help you get behind news stories that use statistics.

Partners

This guide was produced in collaboration with Straight Statistics and the Royal Statistical Society.

What is the problem this guide is addressing?

Statistics can be hyped and sensationalised by the use of an extreme value to make a story more dramatic or by reporting a relative increase in risk without including the absolute change. Data may be analysed and presented in different ways to support contradictory arguments or to reach different conclusions, whether deliberately or by mistake.

But while statistics can be misrepresented, they can also unpick arguments. By knowing the right questions to ask we can discriminate between the proper use of statistics and their misuse. We asked statisticians, journalists and scientists to tell us how they make sense of statistics and what pitfalls to look out for. This guide is not meant to be a lesson in statistics but a source of questions you can ask and pitfalls to avoid. Knowing something about statistics can help you test and debunk arguments and get closer to working out what the figures might be telling us.

Impact

Making Sense of Statistics launched with national press coverage; the guide’s co-editor Nigel Hawkes wrote a blog for the Times and contributor Michael Blastland’s had an article in the Guardian’s Comment is Free section. Since statistics in media are featured prominently in the guide, this was fantastic to see.

The Royal Statistical Society shared Making Sense of Statistics with all of its members and often makes reference to our work in this area. Years later, this remains one of our most popular guides.