Blog

The Big Monitoring Project

There’s a myth in politics that what the Prime Minister says matters. Actually, as he knows, and everybody who wants his job knows, it only matters much if he says it enough times. ‘Long term economic plan’, anyone?

The Big Monitoring Project

So how we will spot and get ahead of all those claims? We need your help to build the tools that can monitor this election properly and level the playing field between big-spending political parties and the rest of us.

Here’s some things we want to cover, and how you can help if you’ve got technical skills –

Television. To pull out subtitles, we need someone to spend some time getting dvbstream, VLC or similar to work with CCExtractor. Not too hard but needs patience and decent linux skills. We also want to save the actual recordings too.

Email. A tool that will scrape an email account, given account details (we’d like to record what the party and candidate mailing lists are saying). Not too hard.

Twitter. A robust scraper for ongoing tweets, given a list of twitter handles and API keys.

Past tweets. A robust scraper for past tweets, given a list of twitter handles and API keys. You can go back about 3,000 tweets with the API.

Facebook. A scraper for facebook pages, if that’s possible.

Pictures. Pictures are going to be important: infographics, twitpics, and other wannabe-viral content. Maybe Tesseract can pull out the text to make them more searchable.

Websites. We think the software archive.org has built might be able to handle this.

What websites look like. You sometimes see full length screen shots of websites. Regular visual records of news websites would be immensely valuable for tracking how the priority things get changes with the news agenda.

Blogs / RSS. Scrape an RSS feed, given a URL.

Thanks to YourNextMP.com we have a decent list of candidates and their URLs to target these tools at. ElectionLeaflets.org and other projects are taking care of tracking things that can’t be automated. But developers — for the things that can be automated, your country needs you.

What difference will it make?

We can’t speak for all the other people who might use this data: journalists, campaigners, researchers, and anyone else who’s curious.

During the election we’re running an 18 hour/day election centre doing rapid analysis of the campaigns’ claims. We’ll use this data to make sure we’re focusing on the things that make most difference, and to make sure that when we’ve spotted something that doesn’t stack up, people know and can choose not to repeat it.

It’s much easier to stop people repeating a mistake than to stop them making it for the first time. We can get the original source to clarify, and journalists can choose not to cover a claim when they see we’ve checked it and aren’t impressed with what we found:

The Big Monitoring Project will let us do that at scale, and make sure the election campaigns are grounded in reality.

Get stuck in

Pick a topic, set up a github repository, create a project and put your code in. We prefer Python 3 but more than that we prefer working code. Have a README file. Have a directory called data/raw which you download the raw content into, data/staging for the processed version, and data/working for anything in between the two.

And what format would we like for the processed version? Here’s an example, a simple adaptation of House of Commons Hansard from mySociety. Basically, each item should be an xml file, with <item> as the root tag, broken into <section>s where necessary (for example if the URL changes or a different person is known to be speaking) and each separate sentence in its own <s> tag — for which we used nltk.

Spin off: how some of the UK’s top research organisations are supporting factchecking

Inaccurate claims aren’t necessarily down to malicious intent. Often, they come about because people don’t understand the evidence or they’re in a hurry. But whatever the reason, inaccurate information does have a negative effect.

So, in 2013 we decided that we wanted to focus on getting the best information out into the public realm, with an eye on making an impact at the 2015 election.

For two years now we’ve been building links with some of the best research organisations in the country. Thanks to that preparation, experts in statistics, research, immigration, health, education and crime will be stepping up to share their knowledge with us and you during the campaign. Imagine an election where the experts are actually heard for once!

Just because they’re experts doesn’t mean they will have all the answers or even necessarily agree with each other. It’s our job to make clear what we don’t know as well as what we do. We’re united by a shared feeling that experts and facts don’t get enough of a look in – and that the upcoming election is the perfect opportunity to change that.

We want to factcheck the 2015 General Election.

We want to factcheck from 6am to midnight, every day, for the six week run up to the election in May.

To do this we need £25,000 to kit out our election hub and make sure we have the manpower to let no claim go unchallenged. Pledge today, here.

What will you actually use the money for?

Since November 2014 we’ve been amassing a small army of volunteers and building links with some of the country’s top research organisations to help us get ready for the election.

But now we can’t actually fit everyone in our office, so 3Space have given us this enormous place on Oxford Street to use during the election period. At the moment it’s empty.

We need to kit out this room, pay volunteers’ expenses and get our experts on the airwaves as much as we can during the election. There are two main costs for us to factcheck the election:

Start-up costs This includes setting up the hub and kitting it out with reliable computers, quick internet lines and monitoring tools.

Day-to-Day costs This includes supporting volunteers and transporting our experts to and from SKY, BBC and ITV so they can tell us the real story! Factchecking for 18 hours a day, 7 days a week, for 6 weeks adds up! Take newspapers for example: each day we need £30 worth of newspapers to stay on top of the claims being made. Over 6 weeks that adds up to £1260.

What have you done already?

We were crucial to getting the Daily Mail and Sun to set up corrections columns. (Yes, they really have them!)

We not only factcheck – we also do something about incorrect claims. We’ve secured corrections from Ministers, government departments, MPs, the BBC, pressure groups and national newspapers.

Factchecking the election is the next logical step. This is what we’ve been building up to!

Why do you factcheck?

76% of people think it’s extremely important that MPs tell the truth but only 26% trust them to do so.

So even when they are telling the truth, many of us switch off. Our votes matter more than that. We need to know what to trust and what not to trust — and who’s got time to check everything themselves?

Full Fact makes it possible to make up your own mind, based on information you can trace. We’ve lost count of how many times we’ve been told “you’re the only ones asking” by a press office — but somebody has to be asking.

And then we go further, getting inaccurate claims corrected and confusing research clarified. When claims get repeated a lot they can do real damage, and that’s something we can stop.

How are you different?

We’re different because we don’t care how you vote. Our work is entirely non-partisan and our agenda is helping everyone have equal access to the facts. As a charity, it’s the law that we aren’t allowed to take political sides.

Our articles don’t dumb down or reduce everything to meaningless soundbites. We look at every side of the argument and present the facts so that you can make up your own mind. We won’t ask you to take our word for it: all our factchecks have links to sources so you can judge the issue for yourself.

What do the rewards mean?

Friends Wall: A page on our website with your name on it. It will pay tribute to everyone who makes factchecking the election possible. We’ll also write your name, or pin up your picture (send it to us at friends@fullfact.org) on our actual wall in the election hub! If you prefer, we can keep your name confidential too – just let us know.

Postcards: “Support grows for the war on error”. Get a pack of post cards and join the fight.

Mug: Put your feet up and relax. Enjoy a cuppa in your very own Full Fact mug.

Tweet: We have over 20 thousand followers on Twitter and we want to tell everyone about your generosity.

Election Party: Join our team and raise a glass as the results roll in and celebrate the most informed election yet!

Adopt-a-fact: You can pick a fact from our database, and we will update you if we do any more work in this area. We’ll use our monitoring system to keep you updated on where it’s popping up and what we’re doing about it.

Post-Game Analysis: A personalised report, and a chance to join our team for a session looking back at how we did, what we learned and what we’ll do for the next referendum or election.

Who will do the factchecking?

Full Fact Staff

First of all there’s our own team, including five researchers who specialise in health & social care, education, immigration & crime, the economy, and the law.

Expert Volunteers

The Office for National Statistics and Ipsos Mori are sending some of their statisticians and researchers to volunteer in the hub, and we’re working with the Royal Statistical Society and others to recruit more individual experts. These people have good knowledge of the topics that will come up, and the sources of information we’ll use to check them – so we’ll be able to factcheck more claims, faster.

Expert Organisations

We’ve been working for two years now to create links with some of the best expert organisations in the country. They have all agreed to work with us during the election – imagine an election where the experts are actually heard for once!

Full Fact Vs Scroobius Pip

While most of Scroobius Pip’s guests on his Distraction Pieces podcast are celebrities, he also likes to interview people “that aren’t as famous but are very much worth our audience hearing”. When he asked us to take part, we were happy to oblige.

The latest episode sees Will and Phoebe spend an hour chatting with the artist and presenter about what we do, why it’s important and how we carry out our factchecking. We look back at some of our achievements to date, and ahead to our ambitious plans for the general election.

In the podcast, Scroobius Pip says: “I’ve always been really glad that I’ve been brought up to be happily proved wrong, enthusiastic to be proved wrong”. We talk about how we try to encourage politicians and journalists to adopt a similar attitude, taking pride in accuracy and making an effort to set the record straight if they’ve made a mistake or used an incorrect source of information.

A gunge tank to jazz things up a bit

Every now and then people come up with great ideas for factchecking. From apps to buzzers to slime, here are some of our favourites:

While we wait for a politician to announce they’re willing to risk being dunked live on TV, we’ve been laying the groundwork to make all these ideas possible.

Apps and tools like this have something in common with live-factchecking, which we do quite a bit of: they rely on a database of checked claims and corresponding verdicts, preferably no more than 120 characters long.

Last month we put out a call for volunteers to help us create this database, plugging in all the claims we’ve ever checked in the past two years, and all the claims politicians have made in speeches and debates from January 2014 to now.

In just four weeks we’ve had 50 applications, 15 new people come to our office and 200 hours of work. It was a daunting task, but on Wednesday we finished it!

Ashley, Steven and Billy — all volunteers — just after finishing off the last entry in the factcheck database.

The next job is to turn this into reusable content: tweets and TV text, 100-word blog posts, and graphics. This will be a vastly superior basis for all our future factchecking across lots of different media – particularly in the run up to the election when thorough preparation will be vital. Please get in touch if:

You’d like to donate to our volunteer operation: we need laptops, chairs, desks and a travel expenses fund. Currently we’re turning away keen volunteers because we have nowhere for them to sit, and no spare computers for them to work on.

You’ve got experience building databases and would like to join in a sprint next year when we transform our raw data into something more searchable and linked up with SayIt.

You want to tell us how you’d use such a database so that when we design it, it’s as useful as it can be for a range of people.

Excel hell

This week sees the launch of a good practice guide for releasing official statistics in spreadsheets. This might not sound important, but if you look behind the uniform face of gov.uk’s statistics hub, you’ll find a bewildering variety of spreadsheets.

After even a short time spent trawling through these inconsistent spreadsheets, HMRC’s betting and gaming bulletin is a relief, with its contents, formatted fonts, and clear contact details.

If you keep clicking through the tabs, you’ll see there’s a rounding policy, a revisions policy, and information about related statistics.

Variety is not the spice of the researcher’s life when it comes to spreadsheet formats.

Small differences between spreadsheets can make our work significantly harder or easier: we have to spend time working out where the statistician’s contact details are, when the statistics were last updated, or whether a number has been rounded. When we’re live-factchecking and trying to get answers out within minutes or seconds of a claim being made, inconsistency among spreadsheets is even more frustrating than usual.

In an effort to lower our blood pressure, we talked to the Government Statistical Service (civil service statisticians) about producing a good practice guide for publishing official statistics in spreadsheets.

Around the same time we also launched a secondment scheme for the Government Statistical Service. Junior staff would come to Full Fact for three months at a time and experience how their statistics are used and interpreted in the wider world, while bringing their expertise to our team.

Our first two secondees were Louisa (from the Department of Health) and Emily (from the Ministry of Justice). They both worked on the project, interviewing journalists and researchers and assessing spreadsheets from different departments to work out what good and bad practice looks like. Read what they said about the project here.

Later on, when we were testing out how useful the draft guide was at official statistics events, we discussed the project with the National Statistician’s Good Practice Team. (The National Statistician is a top ranking civil servant and statisticians throughout government are professionally accountable to him). They took on the task of turning the project into official guidance.

Last week, the guide was launched at the Government Statistical Service’s Northern Conference.

If you regularly handle statistics, you can help by keeping an eye out for changes in the way spreadsheets are published. If you notice they’re becoming more consistent over the next six moths (or, equally, if you think nothing’s changed), please let us know via team@fullfact.org.

See you at the pub?

With the election less than 180 days away, we are preparing to operate an 18 hour a day Election Centre. From there we’ll react rapidly to claims during the campaign and provide free information that can be drawn on by researchers, voters, candidates and journalists.

If you’re interested in getting involved, please come to the Perseverance pub in London (nearest station Russell Square) at 6.30pm on Wednesday 19th November, when we’re holding our second ever volunteers’ event.

We can’t do it alone…

This is the most ambitious project Full Fact has ever undertaken. There’s too much ground to cover by ourselves, so we’re asking for volunteers to:

Research specific topics and restructure our current content for a wider audience

Record claims from politicians and parties as you hear or see them

Build a database where we can store and easily access all this information

Here’s thefull list of topics we’re planning to cover. The more we can prepare now, the more we’ll be able to help during the election itself.

At the 19th November meet up, we’ll be talking about how researchers can help us prepare to tackle these topics and we’d like to discuss our plans with anyone who’s interested in any way. Come along on the 19th or email Mevan to find out more.

General election 2015: what issues should we check?

During the Scottish referendum campaign, Ipsos Mori asked people in Scotland how easy or difficult it was to find trustworthy information. 61% found it fairly or very difficult. We can do better than that.

In the run up to the 2015 general election, we want to create a trove of digestible, robust information about the issues people say they care about, that can be drawn upon by voters, candidates and journalists alike.

We’ve come up with a list of topics we expect we’ll need to be prepared to factcheck, as well as those that might benefit from added context or clarification.

We’re asking you to have a look at our plans, tell us if we’re covering the right things, and let us know if you can help.

If you’re an expert in something we should be covering and are willing to help, please let us know by emailing election@fullfact.org.

If we don’t have trustworthy information at our fingertips, we’re faced with a choice between blind trust and blind cynicism, and it’s no wonder if we disengage. At Full Fact we think this needs to change — and it can only change with your help.