Is there a shy Tory factor in 2015?

What went wrong in 1992

Although I was young at the time, I remember 1992 well. I was allowed to stay up to see the exit polls, and went to bed still curious about what a hung parliament might involve. I woke up to discover that, contrary to all expectations, John Major had won. His popular vote lead was 7.6 points, 9 points (net) better than the final call polls, a polling failure so large that the Market Research Society ordered an inquest into opinion polling, for fear of a meltdown in confidence in market research altogether.

Subsequent investigations revealed that a disproportionate number of those saying they were undecided had voted Conservative in 1987 – many of them ended up doing so once again. There were also faulty demographic quotas (without the frequently-updated ONS updates, pollsters were relying on data from the 1981 census) as well as evidence of a genuine (though small) late swing. But the refusal of some Conservative voters to reveal their intentions was the most troubling because of the obvious difficultly in measuring it, let alone correcting for it.

In the years that followed, we heard talk of shy Tories and the more general concept of the spiral of silence. Labour strategist the late Philip Gould told the Adam Curtis BBC documentary “The Century Of The Self” that his focus groups in early 1992 had discovered people who admitted lying to opinion pollsters. Conservative-commissioned focus groups, Curtis explained, had found the same thing. Indeed, just before the BBC announced their 1997 exit poll, David Dimbleby even joked “we hope they’ve been telling us the truth!”

There were, of course, significant methodological changes in the mid-1990s. But 1992 and (if you care to go back that far) 1970 were not the isolated polling misses that many perceive them to be. 1992 was certainly the worst, but the reality is that elections close enough for polling error to stand out, don’t come around all that often. Largely unnoticed under Tony Blair’s Labour landslides, the shy Tory effect continued long after 1992.

Historical polling error – Perception vs reality

The “everything is fine now” perception may have been reinforced by the exceptional accuracy of exit polls. However, exit polls are completely different exercises, not least because they don’t have the problem of finding a representative sample of likely voters, but instead use a random sample of actual voters. There are no “don’t knows” and they also have clever methods for dealing with refusers. Even in 1992 the exit polls were significantly better than opinion polls (Harris/ITN predicted a 4 point Conservative lead in the popular vote and while the BBC didn’t mention the popular vote from their NOP poll, they had extremely similar seat numbers). There is a detailed writeup on exit polling here.

As for the regular opinion polls, what should we expect them to look like if they were totally unbiased? All polls have a margin of error, a 95% confidence interval within which the true state of public opinion should lie. Individual pollsters aim to be correct to within the headline margin of error of +/-2.2 percentage points for a typical poll of 2,000.

But for polls as a whole, the margin of error is much smaller due to the aggregation increasing the overall sample size. We would also expect errors to be symmetrically distributed, without any discernible pattern. But history shows that the errors seem to have a directional pattern.

I’ve taken a look at the historical evidence, using the polling for which the fieldwork ended in the two days prior to the election (or the final 5 polls where fewer were conducted within that window), for each of the 12 general elections in the last 50 years, using Mark Pack’s excellent database. I didn’t pick 50 years as a round number, but rather because prior to 1966 the fieldwork was either too far in advance of polling day, or not dated, and there were very few pollsters back then anyway.

In 10 of these 12 elections, the Tories were underestimated by the polls, in one (February 1974) the polls were spot on and in one (1983) they were overestimated. Later I’ll show the reasons why this apparent exception is in fact quite consistent with the overall pattern. However it’s also worth noting that the 1983 election was unique in recent times in that over the 12 months preceding it, the government enjoyed a poll lead that was both large (around 13 points) and significantly wider than when it was elected, both of which may be relevant.

What about Labour? Labour have typically been overestimated by pollsters – 9 times in the last 12 elections. Of the three instances of them being underestimated, one of them (1983), is the counterpart to the equal and opposite error in the Conservative vote share mentioned above. The other two (February 1974 and 2010) corresponded with a large overestimation of the Liberal or Lib Dem vote (on the other 10 occassions, the Liberal/Alliance/Lib Dem vote was very accurately estimated, with the polls always within 1.2 points of the outcome.)

In terms of the spread between the Conservative and Labour vote shares, the polls have favoured Labour at 9 out of the last 12 general elections, including 5 of the last 6.

Related

About The Author

NCP founder Matt Singh is the elections and polling analyst that publicly predicted the 2015 industry-wide polling failure the day before. He is based in London and tweets at @MattSingh_. For further information and media inquiries, please see About NCP.

Number Cruncher Politics

Number Cruncher Politics is the non-partisan UK polling and psephology site that predicted the 2015 opinion polling disaster. For more information please see the 'about' page or connect via social media.

RSS Feeds

Disclaimer The information on this site is believed to be correct but no representation or warranty is made as to its accuracy. NO RESPONSIBILITY WILL BE ACCEPTED FOR ANY LOSS OR DAMAGE RESULTING FROM YOUR USE OF THIS SITE.

Our website uses cookies which are pieces of information that is stored on your computer's hard drive and which records how you move your way around a website so that, when you revisit that website, it can present tailored options based on the information stored about your last visit. Cookies can also be used to analyse traffic and for advertising and marketing purposes. Read more about the cookies we use by clicking here. By clicking CONTINUE you agree to cookies being used in accordance with our Cookie Policy. If you don’t agree you can disable cookies – see the Cookie Policy for more details.Continue