Saturday, September 12, 2015

There were two schools of thought in the wake of the extraordinary recent polls from TNS and Ipsos-Mori polls showing a majority for independence. Along with a number of others, I felt it was most likely that there hadn't been any recent change in public opinion, and that what we were seeing was a divergence opening up between online and non-online pollsters. But some people felt that TNS and Ipsos-Mori were picking up a shift that had occurred over the summer, and that the next poll from an online firm would confirm that. Unfortunately, it looks like I was right. In their new poll, YouGov are reporting much the same state of play that they've been showing since their major methodological change a few months ago.

Should Scotland be an independent country?(YouGov)

Yes 48%(+1)No 52% (-1)

So the probability is that the increase in Yes support shown by TNS and Ipsos-Mori was exactly the same increase detected by the online firms in the very early stages after the referendum, and that the position has remained relatively stable since then. If TNS and Ipsos-Mori had been polling on independence regularly over the last year, the chances are that we would have seen a steady Yes lead from them, while the online firms were showing (on average) a very narrow No lead. What's changed over the last couple of weeks is that we're now seeing the whole picture, rather than only half of it. The new information increases the chances that Yes have been in the lead for quite some time without us being aware of it - but we certainly can't be sure of that. It just depends which firms and methodologies you trust most.

The YouGov datasets don't seem to be out yet, but I would advise people to be enormously sceptical of the spin being put on the results of the supplementary questions. This seems to be a classic case of an anti-independence client (the Times) asking a pollster to use leading question wording to produce the results they want. For example, a multi-option question was asked on nuclear weapons to nudge people towards the middle, 'moderate' option, so that backers of the pro-Trident and the middle option could be lumped together to claim that there is a majority in favour of the retention of nuclear weapons. Utterly cynical, and I expect we'll find that equivalent tactics were used to produce an artificial majority against a second independence referendum taking place in the short-to-medium term (a finding that suspiciously contradicts what we've seen from many other polls).

* To put it in perspective, Corbyn has a bigger personal mandate than Tony Blair, who received 57% of the first round vote in 1994 against just two other opponents.

* You'd expect a certain lack of graciousness from Dan Hodges in a situation like this, but it seems odd that he singled out Corbyn's joke about an "Abba tribute band". That strikes me as being exactly the sort of groan-worthy comment that Tony Blair used to make (although admittedly Corbyn's delivery was better).

* Rachel Reeves is leaving the Shadow Cabinet. If today's result achieves nothing other than that, it'll all have been worth it.

Friday, September 11, 2015

There was a real oddity yesterday - a by-election in a ward where there was more than one vacancy, allowing the STV system to come into its own. However, the familiar discrepancy still applied - the Greens were defending one of the two seats in spite of finishing third last time, meaning they had to gain significant ground just to hold what they had. Unfortunately from a pro-independence point of view, they didn't quite make it.

Techincally that worked out as one SNP hold and one Labour "gain" from the Greens, but of course the real story is the 7.6% swing from Labour to the SNP. That's significantly more modest than the swings in other recent by-elections - whether that's the Corbyn effect, or the 'Edinburgh is a different planet' effect, or something else entirely, is difficult to say. Hopefully the Midlothian result will give us some clues.

Excluding a by-election in independent-dominated Orkney, this is also the first time since the general election that any local seat has been won by a party other than the SNP, but in this case it was impossible for them to take both seats - they only put up one candidate.

* * *

I'm still travelling, so have to save my battery, but as you may have seen by now, the swing to the SNP in the Midlothian by-election was very similar to the one in Leith Walk (slightly lower, in fact). Coincidence, or evidence of a Corbyn bounce for Labour? If the latter, we may have to get used to it, because the scale of Sadiq Khan's victory in London today surely makes it inevitable that Jeremy Corbyn will become Labour leader tomorrow morning. Hold on to your hats - we're about to set off on a journey into the unknown.

Wednesday, September 9, 2015

You might remember that when the sensational Ipsos-Mori poll came out last week showing a Yes majority, I suggested that what we really needed now was an independence poll from TNS - the only other firm that uses a 'real world' data collection method (ie. not volunteer online panels) for polling on Scottish affairs. Right on cue, here is the first TNS independence poll since the referendum. The timing is just a coincidence - TNS polls take a long time to conduct, so it was set in train well before the Ipsos-Mori result emerged.

Should Scotland be an independent country?

Yes 53%No 47%

This is an absolutely crucial development. It never seemed likely that the Ipsos-Mori poll was an 'outlier' in the sense that the Yes lead was an illusion caused purely by sampling issues, but if TNS had replicated the small No lead reported recently by the online firms, we might have concluded that Ipsos-Mori's distinctive methodology was producing questionable results. As it is, it looks much more like a straight split has opened up between online and 'real world' firms - with the former mostly showing a No lead and the latter showing a Yes lead.

There are a couple of caveats, however. Firstly, both the Ipsos-Mori and TNS polls were conducted more recently than any of the online independence polls, so it's possible there has been a sudden swing in favour of independence over the last few weeks, and that the next batch of online polls will show much the same thing. I think that's highly unlikely, but the possibility can't be entirely discounted, because neither Ipsos-Mori nor TNS have previously produced post-referendum independence polls, meaning we have no baseline to work from.

Secondly, it's unfortunately the case that it isn't just the 'real world' status of TNS and Ipsos-Mori that distinguish them from the online firms. Neither of them weight their results by recalled referendum vote, which could theoretically explain why their results are so different. However, TNS do (unlike Ipsos-Mori) use a form of past vote weighting, based on the election that ought to be freshest in people's minds - this year's general election. So there's no reason to automatically assume that recalled referendum vote weighting would make a massive difference to the headline figures.

If the divergence really is being caused primarily by data collection method, it could be hugely significant. Intuitively, most of us would probably prefer to put our trust in polls that use a completely fresh sample, rather than relying on respondents who have volunteered to be polled over and over again, and who are more likely to be politically committed. In which case, instinct might lead us to conclude that Yes are more likely to be ahead at the moment.

Although Ipsos-Mori and TNS are showing a very similar state of play, the historical significance of each of the two polls is very different. Ipsos-Mori have never shown support for independence at anything like the current level, whereas TNS are returning to familiar territory in showing a Yes lead - it's hard to believe given their reputation as a No-friendly pollster during the referendum campaign, but if you go back a few years there was a long spell when they were the only firm that occasionally showed a majority for independence, before a sudden and catastrophic drop in Yes support that wasn't replicated (not in full, anyway) by other pollsters. The TNS methodology has evolved over the years, though, so the long-term trend they've shown may be somewhat less meaningful than the one shown by Ipsos-Mori.

TNS have also provided voting intention figures for next year's Scottish Parliament election...

The drop in SNP support may look slightly alarming, but in fact on the list vote it's not significant at all. 51% is identical to the party's share on the list in the TNS poll of two months ago. It looks like the jump to 54% last month was just a quirk caused by sampling variation. On the constituency vote it's harder to say - 58% is (unbelievably) the SNP's worst showing in a TNS poll since the general election, but they're still only 2% lower than in the two polls prior to last month. So we could simply be looking at margin of error 'noise', or the SNP may have slipped slightly - but probably not by as much as the 4% drop suggested by the headline figures.

It's more clear-cut in Labour's case - their share of both the constituency and list vote is significantly higher than in any previous post-election TNS poll, so they probably have genuinely recovered to some extent. There have been some suggestions that this is a 'Kezia bounce', but I'm extremely sceptical about that - I think it's more likely to be the Corbyn effect finally making itself felt. That may come to an abrupt end on Saturday if Andy Burnham or Yvette Cooper upset the odds - if not, the next few months could be highly unpredictable.

It's interesting that the Holyrood figures from Ipsos-Mori last week were interpreted as wonderful for the SNP and dreadful for Labour, because the TNS trend suggests that Ipsos-Mori might have shown the SNP even higher and Labour even lower if they had conducted polls in June or July.

Last but not least, there's severe cause for concern in the TNS poll for the Tories. After Ruth Davidson led her party to a disastrous all-time low of 14.9% at the general election, you might have thought the only way was up - but instead they've slipped further. On the constituency ballot, they've dropped from 15% to 12% since the first post-election TNS poll, and on the list ballot they've slipped from 14% to 11% over the same period. Unless she can turn things around, Davidson may end up looking like the most over-rated political leader since...oooh, Jim Murphy, probably.

We're just three days away from an announcement that should see Jeremy Corbyn become leader of the official opposition (a sentence that I never thought I'd write). But could we be in for a (depressing) surprise? If so, the first sign may come from movements in the betting odds. As you know, I think the idea that betting markets are better predictors than opinion polls is utterly ludicrous - during both the general election and the Labour leadership campaign to date, the markets have followed the polls, and have sometimes lagged quite a way behind. But one scenario in which the markets can be superior is when someone literally knows the result already - for example, the odds moved dramatically in Menzies Campbell's favour just before he was announced as Liberal Democrat leader in 2006.

The result of the Labour deputy leadership contest in 2007 was leaked to the broadcasters a few minutes early, apparently courtesy of an insider adjusting his tie in a particular way, or something of the sort. Having got used to leaks, it was a bit disconcerting to get to the point in 2010 when the Labour leadership result was being read out, and we still didn't have any inkling of who had won. Nick Robinson of course made a prize idiot of himself (not, alas, for the last time) by self-indulgently interrupting the BBC coverage to predict that David Miliband had won. That must be a contender for the worst political prediction of all-time - it's one thing getting the result wrong a few days, hours or even minutes in advance, but to get it wrong while it's actually being read out takes a very special kind of talent. Still, in the light of later events, I suppose we should just be grateful he didn't rewrite the history of his prediction by means of selective editing and a cynical voiceover that he "could have phrased better".

I suspect I'm not alone among SNP supporters in the sense that my head is telling me that an Andy Burnham victory would be best for us, but my heart is making it impossible for me not to want Corbyn to win and consign the New Labour abomination to the dustbin of history. If I was a Labour member/registered supporter, I'd have voted 1) Corbyn, 2) Burnham, 3) Cooper, 4) Kendall. A few weeks ago I'd have given Cooper my second preference, but for reasons only she can explain, she's since positioned herself clearly to the right of Burnham, leading to the recruitment of Dan Hodges as her Number 1 fan.

Apologies that I never got round to fleshing out the previous post about the EU poll. The moment has probably passed now!