A possible solution to the great opinion poll mystery

Ten days on from the election result and we’re still no closer to finding out why the pollsters got it so wrong. Mind you, the mystery was five years old already because the pollsters got it wrong in 2010 too – of which, more later.

There’s a great post about the 2015 debacle on the Understanding Uncertainty blog by David Spiegelhalter of Cambridge University. He shows that while the pre-election polls got the Lib Dem and UKIP vote shares very nearly right, they overestimated Labour support and underestimated Conservative support – which is why the expectation of a dead heat was confounded by a Tory lead of more that six per cent.

Other than a few contrarians relying on gut instinct (such as Dan Hodges, Matthew Parris and my mother) did any of the experts see this coming?

“Not me” admits Professor Spiegelhalter:

“I was convinced by the polls and the electionforecast model, and in a gambling frenzy even put £1 on a Lab-Lib pact at what I thought were generous odds of 8-1. William Hill were offering 9-1 against a Conservative majority, so the betting markets were taken in too, and I usually trust them…”

There was at least one polling expert willing to contradict the conventional wisdom (before the event, that is):

“A lone public cautionary voice came from Matt Singh of Number Cruncher Politics, who the day before the election wrote on whether there was a ‘shy Tory’ effect. Based on detailed models of past election predictions and outcomes, he concluded that 2010 was an anomaly and Conservative voting tended to be under-estimated in polls…”

2010 is certainly a problem for the ‘shy Tory’ theory, because the pre-election polls five years ago got pretty close to the actual Conservative vote share. Furthermore they only slightly underestimated the Labour vote share. What they got badly wrong, however, was the Lib Dem vote share, which was significantly overestimated.

Unless there’s been a major change in polling methodology between 2010 and 2015, any explanation for the 2015 discrepancies also has to explain the very different 2010 discrepancies. So, on the basis of precisely zero expertise in this area, I’d like to offer a hypothesis of my own.

I start from the assumption that the public are getting wise to the fact that they can use the polling process for their own purposes. This is something I’ve written about before:

“…by ‘polling process’ I’m not referring to the actual procedures of the polling organisation, but the subjective experience of those being polled. For instance, some respondents might perceive the process as a mild form of inquisition in which their opinions are being scrutinised, while others view it as a platform for getting their views across…

“…Whether consciously or subconsciously, the public is getting savvy about the role that polling plays in setting the political agenda…”

I would further suggest that while there is a subset of individuals across the political spectrum who use their participation in polls to send a signal, there are important left/right differences in the purpose (and not just the content) of this signalling:

Those of leftwing or liberal disposition tend to be ‘virtue signallers’ (to adapt James Bartholomew’s terminology) – i.e. people who feel the need to advertise their allegiance to what they believe to be the ‘correct’ values and opinions. By way of contrast, small-c conservative types tend to be what I call ‘disappointment signallers’ – i.e. people who don’t feel the need to make a show of their principles, but who do wish to express their dissatisfaction with those they judge to have let them down.

In both cases the signalling doesn’t necessarily translate into action – i.e. an expressed voting preference fails to become an actual vote because the signaller either votes another way or doesn’t vote at all.

These assumptions can be applied to explain both the 2010 and the 2015 polling discrepancies. In 2010, the Lib Dems were the prime attractor for the virtue signallers. Indeed, “I agree with Nick” – a quintessential piece of virtue signalling – became the most potent phrase of the entire election campaign. On the other hand, disappointment signalling was not a very strong force in 2010. After thirteen years in opposition, the Conservatives weren’t really in a position to let their supporters down (apart from the hardcore rightwing ideologues who had already quit for other parties).

2015 was a very different election in which the virtue signallers of the left abandoned the Lib Dems for Labour (and the Greens), while the disappointment signallers of the right had a full five years of Conservative-led government to feel let down by (and were therefore inclined to signal their disappointment to the pollsters by claiming they’d vote for another party or stay at home).

Because of the crucial gap between signalling and action, all of this translated into the Lib Dem overestimate in the 2010 polls and the Labour overestimate / Conservative underestimate in 2015.

I’m not saying that this is the only explanation for what went wrong, but I certainly think it’s part of the solution. Then again, as someone who believed the polls were right, you shouldn’t take my word for it…