This website is undergoing reconstruction - please forgive any issues!

Shape of the polls

Something never sat right with me in the run up to the 2015 general election. The polling showed Labour doing well, yet you would rarely hear a good word said about Ed Miliband. No-one seemed to think he was Prime Ministerial, or that he was a good leader, or that he would be capable of running the country. Despite the parliamentary reality, the way people think about voting tends to be more presidential – based on who they want to be Prime Minister, rather than their constituency MP. So why was Labour doing so well in the polls?

In the end something had to give, and it turned out to be the polling data. Given that polling of voting intention has proved unreliable twice in recent memory now, maybe there’s a better way to present political opinion polling? Here’s what I came up with:

Satisfaction with party leaders levels have correctly predicted the outcome of the general election each time since polling on the matter began (see below). Adding leadership satisfaction to the mix allows distinction during circumstances that might otherwise be described as neck-and-neck (the 1991-92 data is a prime example – the voting intention numbers are roughly the same, but the leadership satisfaction ratings reveal a huge gulf, one which makes the result of the 1992 election far less surprising). Leadership satisfaction polling is also probably more reliable than voting intention – it’s not subject to any certainty to vote issues, for instance.

I also thought it would be helpful not just to chart position, but also provide an indicator as to whether a party is in a position where it is likely to win the next election. Using the results at each general election (see below), I’ve highlighted a “Winning Zone”, outside of which no party has ever won a general election.

Given the humiliation of pollsters in May, maybe it’s time for a rethink of how we display polling?

3 Comments

Paddy Alton

Not sure if it’s a better way to *display* polling. After all, all the polls for 2015 fall outside the winning zone. It certainly holds some lessons for how we evaluate polls though – the boundaries of the winning zone are clearly interesting, since they would have told us the Conservatives were in the right place to win, and Labour weren’t, even if the voting intention polls weren’t saying that.

I guess what you’re suggesting is that in actual fact the votes the two main parties receive are correlated well with the leader approval (NB the axis on the last plot is mislabelled – should be vote share, not voting intention), which looks reasonable. We implicitly expect that voting intention will also be extremely well correlated with actual vote share. If the two measures are giving different answers, that therefore indicates a problem.

(Incidentally, I wonder whether the thing to look at is really whether ‘winning margin’ — the difference between the vote shares — is correlated with ‘leadership gap’. Seems to me the 2010 results – when the Lib Dem share was a factor – are scattered off an otherwise rather tight sequence…)

I suppose one option would be to draw a nice straight line through those points, derive an uncertainty region around them, and then in future combine that with the leader approval and its polling uncertainties to produce a predicted outcome. The uncertainties on that outcome might be larger than we want, but perhaps that could be brought down by combining with other measures like ‘who do you trust with the economy more’. In principle that could be potentially used to diagnose VI polling failures in advance…

Paddy Alton

Displaying the winning zone isn’t particularly helpful, I think: all the polls for 2015 fall outside it. The boundaries clearly *are* interesting: in this case they tell us that even if the polls for VI are neck-and-neck, one party is in the correct place in terms of leadership and the other isn’t. I’m not sure this is so much a lesson for how polls are displayed as it is for how they are evaluated by the people who conduct them.

What (it seems to me) you’re *really* implying is that there is a correlation between votes cast and leader approval; indeed, your final graph shows this nicely (although the x-axis is mislabelled – should be votes cast, not voting intention). Likewise, a poll measures voting intention, which (we hope) is strongly correlated with the way people vote, and therefore seen as useful. If the two measures are giving different answers (like in 2015), we know something is up.

What we *could* have done is drawn a nice straight line through the points on the final graph, quantified the scatter, and drawn confidence regions on the plot. Combined with polls on leader approval (and their statistical uncertainties), the prediction with maximum likelihood would in fact have not been far from the actual result. Not sure the uncertainty would be small, but perhaps combined with other measures – e.g. ‘who will be best for the economy?’ – that could be brought down.

(I also wonder whether a differential approach might be more robust? In other words, is it possibly the case that the margin of victory correlates more strongly with the ‘leadership gap’ than the absolute result correlates with leader approval? If that needs motivation, consider 2010 where the two results are scattered left from the general trend – due to a good showing by the Lib Dems, perhaps?)