What Do We Know about the EU Referendum?

Just in case you missed it: a little while ago we had an itty bitty referendum on whether the UK should stay as a part of the EU.

Given that this has had a small, barely worth talking about really, hardly noticed it impact on British politics, we wondered whether there would be any visible changes in the way that people are using our Freedom of Information site WhatDoTheyKnow.

Did people suddenly find themselves wanting to know more about Europe-related matters in the run-up to the referendum? What about afterwards?

Short answer: Yes they did! To both questions!

Long answer: Same as the short answer…but with graphs!

What we did

First we drew up a list of twenty-three keywords which might indicate that the request was at least partly related to either Europe, the EU, or the topics that became part of the debate leading up to the vote: keywords like EU, European Parliament, Schengen, refugee, and, that brave little neologism that could, Brexit*.

Then we pulled all requests where the requester had used one or more of those phrases** and started number-crunching.

What we found

In the period between the May 2015 general election and the June 2016 EU referendum WhatDoTheyKnow sent 1,022 FOI requests that matched our EU keywords. These were generated by 641 unique requesters.

Looking at these requesters: 79% of them made just a single request, and 96% made four or less. The remaining 25 users made 25% of all EU requests — with three users making more than 20 requests each.

For the year leading up the election there was an average of 55 users making 75.6 EU-relatedrequests between them each month.

If we split this into two halves (the last half of 2015 and the first half of 2016), the average number of users per month had increased by 20 in 2016 compared to the second half of 2015 — with a peak in both users and requests in the month before the referendum and a decline in the immediate run-up.

So people had more questions to ask once the referendum was more in the public eye. But maybe that’s just reflecting wider trends across the board. Can we state with certainty that this change was referendum-related?

Let’s move on to the second question: What happened after the referendum?

After the referendum

Comparing the three months before the referendum with the three months after it, we see users and requests are up in the post referendum period.

EU-related requests

Users making EU-related requests

Pre-referendum

310

216

Post-referendum

332

252

Looking month-by-month, we can see this is mostly an immediate spike followed by a drop-off:

In fact when we looked week-by-week, we could see the largest spike was in the week following the vote. This gives us some definite hints that it was the referendum that was driving this.

But to make extra sure that this increase really was referendum-related, we compared these changes to the overall WhatDoTheyKnow trends at the time.

The number of requests made across the platform increased between the two periods (17,246 increased to 19,120) — but there was also a decrease in the number of unique users making requests (4,850 decreased to 4,721).

This means the post-referendum increase in EU requests was counter to the general flow – and we can use a statistical test (chi-square) to confirm that the difference in users making EU requests is sufficiently different from the overall direction of users to reject the idea they are being driven by the same trend (p < 0.01 for those that want to know) .

So we can say there is a real difference before and after the referendum: people were asking government for more for more EU-related information after the referendum than before it.

Notes

*First appearance in an FOI request: May 2015!

**Obvious Complaint: But Alex! Aren’t some of those a bit broad? And the answer is yes! In fact we discarded ‘immigration’ and ‘migration’ as keywords because when separated from other keywords, these were mostly requests for information about immigration rules relevant to the requester (although that said, a similar post-referendum peak appears when we looked at these ‘immigration’ requests in isolation. There were just too few to make as big a deal out of the change).

‘EU’ as a keyword will similarly be catching requests that have nothing to do with the EU, as EU law is so integrated that appeals to directives or other obligations can make an appearance in requests to just about any public body on just about every topic.

While the global count of ‘EU related requests’ might be inflated by this, a change relative to the population of all requests (like the one we found) should be robust — assuming that non EU-related requests that mention the EU are not distributed differently to non EU-related requests that don’t. This seems reasonable and so for the sake of this blog post — let’s say that’s so.

Keywords used

Here are the words we used (note on why we didn’t include ‘immigration’ or ‘migration’ above); one request often matched multiple keywords:

mySociety

mySociety is a not-for-profit social enterprise, based in the UK but working with partners internationally. We build and share digital technologies that give people the power to get things changed, across the three areas of Democracy, Freedom of Information, and Better Cities.