Shaping Society

What happens when we don’t have good data?

Amy Sippitt is Full Fact‘s research and impact manager. She runs a team of fact-checkers, and promotes high-quality research into the impact of fact-checking and the misinformation ecosystem.

The Need to Know project was launched in February to anticipate and plan for what information is needed for upcoming public decisions. Here Amy — who co-ordinates the project — explains more about what the project hopes to achieve.

Experts can and do work together to call out spurious factual claims and argument. But they also play a big role in laying the groundwork for debate. This starts with attempting to predict the big debates that will be happening in five years’ time, and producing information to inform these debates before things get too heated for the information to be heard.

As Full Fact’s Director Will Moy said at the kick-off event last month: “If the EU referendum taught us anything, it’s that expertise and experts should be set out as part of the furniture of public debate early on if they’re going to have an impact.”

“We need to start now so we’ve got the data and analysis we need to answer the public’s and other decision-makers’ questions before the heat of the next election.”

Need to Know’s first working session in February asked academics, journalists, fact-checkers and statisticians to answer three questions:

Have we identified what decision-makers (including us voters) want to know?

The topic focuses for the working session were based on Ipsos MORI’s issues index. Every month, pollsters Ipsos MORI ask people “What do you see as the most important issues facing Britain today?” Then they aggregate the answers into categories. Here is February’s index:

Taken from Ipsos MORI Issues Index: February 2017

Participants at the workshop identified what the major areas of public concern are in their field, and what key choices will need to be made. We asked them to think about issues that affect a smaller group of people a lot (like support for children with special educational needs), and issues that might affect larger groups of people (like how to pay for social care in the future).

Have we got the information we need to properly inform those debates?

We’ve got a lot of great data and analysis. Our participants identified that we have some pretty good answers to questions like how different types of schools perform (ie grammars, academies vs other), how Brexit might affect the NHS workforce, and whether traditional crime is going up or down (ie non-cyber).

But still it’s really hard to definitively answer some of the most interesting and important questions public debate throws up: what’s the impact of immigration on schools and the NHS? How long are people waiting to get seen by a doctor? What’s the impact of trade deals on wages and prices?

There are many areas where we only know the average impact, rather than how things are affecting individuals.

Just after the EU referendum, Bank of England Chief Economist Andy Haldane gave his ‘Whose recovery?’ speech (PDF) in Port Talbot in Wales. He noted that: “This has been an uneven economic recovery, looking across regions, income and age cohorts. Large parts of the UK — many regions, those on lower incomes, the young, renters — have not experienced any meaningful recovery in their incomes or in their wealth.”

He warned that “aggregate activity measures are sometimes a poor proxy for the average person’s income”.

Where that information exists, are we explaining it properly?

Participants discussed whether presentation and communication are up to scratch. Do journalists and people reading about these issues have the same understanding of particular terms or concepts as the people who produce the data?

Take homelessness, for example. Not everyone who becomes homeless is entitled to be housed. People who are legally homeless are “rarely” on the streets, according to the Department for Communities and Local Government. We don’t have definitive statistics for how many people are sleeping rough — and the availability of detailed data varies by area.

What’s next?

We’re working with the three other partners — the ESRC, the UK Statistics Authority, and the House of Commons Library — to identify what we can do to address these gaps. We’ll be announcing more on this soon.