New NCP analysis – where the polls went wrong

New analysis published today by Number Cruncher Politics finds that the polls were wrong at May’s general election because the flows of voters between parties, following the Lib Dem collapse and the UKIP surge, were substantially different from what voting intention polling had measured.

By modelling electoral data and using the cross sectional (face-to-face) wave of the British Election Study (published on Friday), NCP is now able to explain what happened.

Polls had almost without exception shown that 2010 Lib Dem voters that had switched to one of the two larger parties had overwhelmingly opted for Labour. However, the analysis suggests a different picture – Lib Dem to Labour switchers did outnumber Lib Dem to Conservative switchers, but only narrowly.

Polls also consistently underestimated the UKIP threat to Labour. On average, they got the UKIP vote share right, but badly mismeasured where it came from. Almost all had shown proportions of 2010 Tories to 2010 Labour voters to be two- or three-to-one – but the new data suggests that the breakdown was significantly more even.

It is likely that over the last two election cycles, UKIP has taken more voters from Labour than from the Conservatives (or any other party).

Polls had also suggested that the direct flows between Labour and the Conservatives were very small and essentially cancelled each other out. There was in fact a very small net flow of 2010 Labour voters to the Tories.

Click to enlarge

These errors were reflected elsewhere in polls besides voting intention. 2015 Tory voters appear to be much more socially liberal – with more progressive views on gender and racial equality – than was previously thought.

The new BES data also suggests that the Tories had both a larger vote share and a larger lead among women than among men.

These numbers support the theory that Conservative modernisation helped – not hurt – the Tory performance in 2015, as Daniel Finkelstein argues in today’s Times, citing the new NCP analysis.

Methodologically, the primary root of the problem is very likely to have been a sample imbalance combined with problems weighting the data.

It is widely noted that opinion poll respondents tend to be more politically engaged than voters as a whole. However, this bias also arises unevenly within opinion poll respondents. Further analysis of the sample and its imbalance has identified a significantly undersampled section of the electorate, which was Conservative-learning in both 2010 and 2015.

Those that had voted for one of the coalition parties in 2010 and subsequently changed their vote to an opposition party were overrepresented in polls. The risks from this “electoral flux” were highlighted in NCP’s pre-election analysis on May 6th, which predicted the polling failure and Conservative victory.

NCP has under development a model based on this new analysis, to attempt to solve the underlying problems. The adjustments from the initial version of the model correct around 4 points of the error (or around 5 points taking into account differential likelihood to vote). The average of published polls at the election understated the Conservative lead by 6.5 points.

Notes: Number Cruncher Politics (NCP) is the non-partisan UK polling and psephology site that predicted the 2015 opinion polling disaster. For more information please see the ‘about’ page or connect via social media. Matt Singh is a polling analyst and NCP founder. He is based in London and tweets at @MattSingh_.

Related

About The Author

Number Cruncher Politics is a non-partisan blog focussed on UK psephology, statistical analysis, opinion polls and politics. For more information please see the 'about' page or connect via social media.

Number Cruncher Politics

Number Cruncher Politics is the non-partisan UK polling and psephology site that predicted the 2015 opinion polling disaster. For more information please see the 'about' page or connect via social media.

RSS Feeds

Disclaimer The information on this site is believed to be correct but no representation or warranty is made as to its accuracy. NO RESPONSIBILITY WILL BE ACCEPTED FOR ANY LOSS OR DAMAGE RESULTING FROM YOUR USE OF THIS SITE.

Our website uses cookies which are pieces of information that is stored on your computer's hard drive and which records how you move your way around a website so that, when you revisit that website, it can present tailored options based on the information stored about your last visit. Cookies can also be used to analyse traffic and for advertising and marketing purposes. Read more about the cookies we use by clicking here. By clicking CONTINUE you agree to cookies being used in accordance with our Cookie Policy. If you don’t agree you can disable cookies – see the Cookie Policy for more details.Continue