Stuff

This morning’s Mail on Sunday had a new poll of South Thanet which they built up into a UKIP “covering up” an unfavourable poll showing them headed for defeat. ComRes have subsequently released the tables for the poll here, revealing it was actually commissioned by ChartwellPolitical, an agency founded by two former UKIP staffers.

First let’s cut away the Mail’s hyperbole – it’s really not a “LOSER POLL!” and doesn’t show Farage heading for a humiliating defeat. UKIP are one point behind the Conservatives, with Labour one point behind them – CON 31%, UKIP 30%, LAB 29%, LDEM 5%. Given the margin of error, one couldn’t confidently say which of the three parties are ahead. What it actually shows is an extremely tight race. However it’s significantly less positive for UKIP than the previous polling in Thanet South – a Survation poll back in February that showed UKIP ten points ahead, and it resulted in UKIP attempts to rubbish the poll and its methodology last night.

In terms of methodology, the poll is mostly done using the same methods ComRes use in their national telephone polls – same turnout weighting and filtering, same squeeze question, same treatment of don’t knows. There are two important differences between the way ComRes do their national and constituency polls though. First respondents were prompted with the individual candidate names, secondly the poll was NOT politically weighted (if it had been, it would probably have been better for the Tories to some degree, depending on how much false recall ComRes allowed for in setting targets).

Most of the criticism of the poll last night (including some from UKIP themselves) was frankly complete nonsense. I can only assume a lot of it was sourced from “what some bloke on Twitter reckoned”. To sum up, the difference isn’t because candidates weren’t named – they were. It isn’t because 2010 political or turnout weights were used – they weren’t. It wasn’t because people who are unlikely to vote were included – they weren’t. It wasn’t because ComRes reallocated people by 2010 vote – they don’t. The idea that the question wording mentioning “your local MP” favoured Laura Sandys seems somewhat stretched, given the question included candidate names and Laura Sandys wasn’t one of those candidates.

To look at the more substantive things people have asked though, the initial voting intention question in the poll found Nigel Farage ahead. What put him behind in the final figures was weighting by turnout and squeezing the don’t knows. Neither of these are strange and unusual, they are ComRes’s normal method and are perfectly justifiable.

Looking at turnout first, ComRes found that Labour and Conservative voters said they were more likely to vote than UKIP voters. In Survation’s poll in February they actually found just the same thing, and their approach to weighting by likelihood to vote is very similar to ComRes’s (there is a difference in how they treat people who are very unlikely to vote – Survation weight them down very heavily, ComRes exclude them. In practice this difference has minimal effect). The difference between the Survation poll showing a ten point UKIP lead and the ComRes one showing a one point Tory lead is NOT turnout weighting.

The other difference is don’t knows. In the Survation poll people who said they didn’t know how they would vote were ignored. In ComRes, they were asked a follow up “squeeze” question – how would they vote if they legally HAD to. For people who still didn’t give an answer, ComRes asked if they identified with any party, and took that as their most likely vote. In practice these squeeze questions helped Labour and the Conservatives, but didn’t squeeze out much in the way of extra UKIP support.

There is nothing at all methodologically “wrong” with this poll… but then, there wasn’t anything “wrong” with the Survation poll in February either. There are different methodological approaches, and there are good arguments to be made for and against them, but we don’t have the evidence to say which is right. More importantly, a lot of the difference here isn’t because of methodology… it’s just because ComRes found fewer Ukippers and more Labour and Conservative voters than Survation did. Perhaps that’s because UKIP have lost support since February. Perhaps that’s just normal sample variation. We can’t tell, we can only say that South Thanet may still be a tight race after all.

748 Responses to “ComRes poll of Thanet South”

Survation have shown a consistent UKIP bias (‘bias’ in the non-partisan sense) in all of their polls this year, from what I can tell. Either their methodology is bang on the money and everyone else is wrong, or…

The use of privately commissioned polls by local parties is fraught with issues. Local here in Hornsey and Wood Green the Lib Dems are known have commission a series of these but they chose to only publish one. This showed them a point behind Labour (they are defending this seat). I feel fairly confident that the released poll was the best for them and now they are putting out publicity saying it is ‘neck and neck’. So what are we to make of all this? Probably that genuine opinion surveys carried out by reputable polling companies are used purely for party advantage. Bad news for the polling companies I would say and also bad news for the general public who think they are getting honest accounts of the election.

I don’t think any of the models really apply in these leadership seats, so I will say the same as I said for the Sheffield Hallam polls – too much uncertainty around those don’t knows, so the poll can’t really tell us much other than advantage UKIP so far, but it could go any way as we don’t know how the don’t knows will really vote.

Isn’t it more probable that over55s who haven’t voted before (or, at least, not for some time) are less likely to vote now, than youngsters who haven’t voted before , but who are more likely to do so as they get older?

It may happen that you put your bet on the right person as far as the polls go (all to play for). It’s really your private business.

But, just to make the point to you as a statistician. Polling is frequentist statistics that you happily marry with invalid Bayesian (as you are attributing chances to the hypothesis) statistics. That’s pretty bad.

@Lazlo I think both frequentist or Bayesian approaches to predictions miss the point. The campaigns both national and local will affect the result and it is just too early (and the figures are too close) to use the opinion poll to get a valid prediction.

Survation have shown a consistent UKIP bias (‘bias’ in the non-partisan sense) in all of their polls this year, from what I can tell. Either their methodology is bang on the money and everyone else is wrong, or…

To be fair you’re really talking about different things. Survation’s ‘regular GB-wide polls (now for the Mirror) are online one and weighted in a way that Richard has shown produces a different social mix from other polls. This mix is very C2-heavy which in turn helps UKIP whose greatest strength is in that group.

Their constituency polls are necessarily done by phone and Survation only weight them by “age, sex and ward”[1]:

so neither income nor SEG is used – though arguably something should be. So the pro-UKIP bias[2] we have consistently seen in Survation’s national polls should not be the cause of the same bias in their constituency ones.

[1] Neither Ashcroft nor (as far as I can see) ComRes seem to weight by ward or any other geographic marker (such as postcode) within constituencies – though it’s possible they may use them in quota sampling. It’s possible that in constituencies made up of several distinct and perhaps politically divergent communities this could be a problem

[2] Which doesn’t mean they are wrong, but practically everyone else will be if they are right.

I do find this kind of stuff amusing. There are too many people who simply cannot accept a poll (some actual empirical research) that disagrees with their opinion. These tend to be the same people who don’t understand what a margin of error is, or who accuse those who produce national forecasts of bias is the largest party happens to be one that the pollster has links to (e.g. Stephan Shakespeare and the Conservatives). The fact that all the data tables are public access and methodology clearly published is besides the point – the result is not what we wanted to it must be biased.

You see this all the time in popular level responses to statistical scientific research – if a research result doesn’t fit in with your dogma then clearly the research, not the dogma, must be wrong.

On another note, my EWMA analysis (http://forecastuk.org.uk/2015/03/31/updated-ewma-analysis/ for the last published) now shows a new Conservative mean around 34.75, the likelihood of a new Lib Dem mean around 8 (one UCL breach already, if tomorrow is the same then confirmed) and a new Green mean around 4.5 (again, one LCL breach already, if tomorrow is same then confirmed). Will try and publish new charts in next day or so.

@Andy Shadrack – “What percentage of UK citizens would be immigrants? What percentage of UK citizens would be sons and daughters of immigrants or have immigrant grandparents?”

Quite a lot – as far back as 1701 Daniel Defoe was talking about how mongrel the English were. This has always been a class-based society rather than a race based one.

However, there is also a class element to the mixing. The higher you go the more mixing there is. Aristocrats have always been keen to marry foreign heiresses – they not only bring in money, but clever genes, helping the family to maintain it’s grip at the top of society, and prevent the horrors of in-breeding. There were hundreds of American heiresses at the start of the 20th century who married into the British aristocracy, like Consuelo Iznaga a Cuban-American who became the Duchess of Manchester and Churchill’s mother, Jessie Jerome, she was the daughter of a self-made Wall Street speculator. Then there were all those escapees from revolutions elsewhere – Clegg for example is descended from Russian aristocrats.

Then of course empire played it’s role too – look at the number of middle and upper classes with an Indian ancestor, eg Prince William via Diana and the Earl of Liverpool (prime minister from 1812 to 1827) to the likes of Billy Connelly and Alistair McGowan.

Then you’ve got the waves of immigrants into London in the late 19th and early 20th century, Germans, Jews, French etc. They all left their mark on Londoners.

The least mixed are provincial working class people, especially those who refused to move very far to look for work, living and marrying locally in a small pool. It’s possible that Farage is really trying to reach those people.

I don’t see how anyone could draw any other conclusion that it’s a tight race thereabouts and any of the three highest placed parties could win. But then there’s newspapers that need selling and exciting headlines that need printing.

Not for the first time, I find myself wondering if there might be a “shy UKIPer” situation going on here or elsewhere, in the same way that we thought we’d resolved since 1992. Granted that many UKIP voters are ex-Conservative voters, we could assume they are in this sense accounted for already. But what about ex-Labour voters? Would they automatically be less “shy” simply because they transferred their allegiance from a party that never had this problem?

Or is there something about being a UKIP voter which tends to adjust your certainty to vote without regard to previous allegiance? A sort of SDP-style new party syndrome, perhaps.

In the 2010 election the actual votes were:
Tory 48%
Labour 31.4%
Tories were bigger by a factor of 1.53:1

In this survey, the raw data asked for 2010 vote:
Tory 28%
Labour 22%
The starting point is 1.27:1… A lot of adjustment required…

Indeed it is. Because you need to look at the percentages after the non-voters etc have been removed[1]. This gives the recalled 2010 vote as:

Con 42.6% (48.0)

Lab 33.8% (31.4)

Lib Dem 14.3% (15.1)

UKIP 9.2% (5.5)

(Actual 2010 result in brackets – only these four Parties stood)

As you can see there’s not a great deal of difference except people are more likely to say they supported a Party that has increased in popularity since and less likely one that has decreased. Some pollsters make an adjustment to compensate for this which, as Anthony hints, would probably have meant the targets they were adjusting to were even nearer to the figures they got.

[1] Strictly speaking neither these nor the figures you quote are ‘raw’ because they have been adjusted to take account of age etc.

I think you’re right. It’s likely that we actually *don’t* know what the UKIP support will be like until the votes are counted (which I know is obvious, but you know what I mean in relation to polling). The pollsters have tried their best to handle the issues around the main parties since 1992 (and remember the polling issues there were not just “shy Tories” but some qualitative evidence that people made up their minds on the day, perhaps even in the ballot box – “If Kinnock wins, will the last person…”). I wonder whether this is the first election since then that the same thing is going to happen. I think people will actually make their minds up about voting for (or not) UKIP on the day itself and that decision itself could be driven by last minute events. That kind of voter decision making is difficult to poll a month out from the vote.

There is a case to be made that for a constituency poll past vote weighting is actually not really important – you want to try and get a properly weighted sample on the constituency demographics – previous voting intention might actually skew the result away from the reality on the ground.

Candy
However mixed the current inhabitants of the UK are, there has never previously been immigration on the scale that there is now. This causes unprecedented pressure on services such as schools and hospitals. Hence UKIP’s popularity. It’s nothing to do with race, simply to do with the country being swamped.

Yes, there might be a class element to it, because wealthier people can avoid some of the hardships by paying for private education and medicine for instance. This option is not available to poorer people who see their children being unable to buy houses, and themselves having to wait longer and longer in hospitals.
—–
Other posters have mentioned a possible ‘shy UKIPper’ phenomenon. This is quite possible as there have been incidents of UKIP supporters (and leaders!) being attacked.

You see this all the time in popular level responses to statistical scientific research – if a research result doesn’t fit in with your dogma then clearly the research, not the dogma, must be wrong.

—

Good post.

I dont understand either statistics or polling methodology. Yet I accept their findings without question. With the one proviso – as we are always being advised – to look at the trend not the invididual poll.

That said, it ought not to be forgotten that there are many cases of ‘science’ being dogmatic too. Its something to do with being human I suspect; as a species we seem to believe that if we look long enough for something we’ll find it. Whatever it is.

Most of them show a three way split and it’s interesting how Labour have managed to hang on to their vote in the last year. They might have hoped to use the special circumstances to squeeze Lib Dem and Green votes more – the split of 2010 Lib Dems (small sample 72) is:

There’s a campaign of local people (just recently getting off the ground) to keep Farage out.

This poll may have given the local campaigners something to work with; & they may be willing to unite behind the Labour candidate (because they are anti-UKIP & not very keen on the Tories) & Labour appears to have a decent amount of existing support.

There is a case to be made that for a constituency poll past vote weighting is actually not really important – you want to try and get a properly weighted sample on the constituency demographics – previous voting intention might actually skew the result away from the reality on the ground.

There’s a good case to be made for that, though if you find recalled vote is going too far from what you would expect there may be something wrong as well. There seems to be a particular problem in Scotland because of the shift to SNP in 2011 and people misremembering that as their vote – though using 2011 has its problems too, especially for those who didn’t vote in 2011 but did in 2010.

How you handle 2010 non-voters can also give large variation between pollsters as well – and that’s something that can get ignored in these weighting discussions.

the highest proportion of mixed relationships are to be found amongst the working class (social economic group d and e) – if you think about were you have the most diverse communities – its in working class urban areas as opposed to the leafy suburbs.

Id also disupute the ‘immigration = pressure on services” meme – big population increases may increase pressure on services in the south east – but the pulls there is economic factors. In addition those with the greatest demands on – say – health services – are elderly people. most immigrants are young – indeed many immigrants moving into the south east could very well be employed in health services or transport – so they are actually helping relieve some of the demand on services.

Its interesting to note that areas with very high levels of immigration tend to have very low support for racist/xenphobic parties – there support tends to be highest which are very ‘white’ but not affluent. So its amongst people who have little direct interaction with ‘immigrants’ – and who are less likely to be affected by the effects of immigration.

@Amber Star – a few days ago Mike Smithson was saying that UKIP were getting troubled by Labour in Thanet. He had been told that they felt they had done enough to see on Cons, but that Labour had a good candidate and could come through the middle to win the seat. This was well before this poll.

Your EWMA analyses and interesting and seem to provide a very sensitive instrument for picking up changes.

Two questions, if you have time to respond:

(1) How are new mean scores determined? From your graphs the process seems very odd. For example, the LibDem start out on 12th January already almost breaching the LCL. Has the 12 Jan mean been inherited from an earlier phase that is not showing in your graph? Also, how is the mean re-set when you have a double breach and start a new phase of analysis? In the Labour graph I can’t understand why on Feb 5th the new mean is set as high as 34%. On that date – and indeed for another month – the rolling average had never been that high. What calculation was used to set the Feb 5 average at 34%?

(2) Is it possible to adapt this method to answer the question: Has Tory VI risen above Labour VI? It seems that you might be able to do this by tracking (say). Tory VI minus Labour VI, and presenting the designated mean to zero (rather than letting any algorthm set it for you). That way, you could use standard methods to track when the measure breaches the confidence boundary twice, and so declare that one party has moved ahead of the other. Would that be viable?

One point – if it’s so close, confusion with FUKP could well cost UKIP vital votes. We all remember the Literal Democrat. Just what Al Murray wants, of course: behind the comedy there is serious intent to keep Farage out.

I have heard similar stuff to Mike Smithson; & the local campaign against UKIP & its EDL camp-followers may coalesce around the personable Labour candidate – especially now this poll shows Labour probably has a good chance of winning.

Unicorn – the mean and standard deviations are derived from the last five polls at the point of the breach of a LCL or UCL. So yes, if the poll ratings are consecutively higher over five days, the first EWMA of a new series could already be breaching the limits of the new mean point.

I think the idea of a Conservative minus Labour EWMA is genius. Let me put the kids to bed and then calculate it for you and publish it.