Saturday, May 21, 2016

When I posted about the EU referendum polls yesterday (and updated this blog's Poll of Polls), I somehow managed to miss the big polling story of the day - that YouGov had conducted side-by-side online and phone polls to test a theory about what might be causing the discrepancy between the findings produced by each method, and that both polls had shown a small Leave lead. This is just the second time in the whole campaign that a phone poll has reported an outright lead for Leave. It only seems to have happened because YouGov were especially persistent in trying to contact potential telephone respondents, and as a result eventually managed to speak to enough non-graduates, who are more likely to be Leave voters. Bizarrely, at almost exactly the same moment that these findings were released, the very recently-retired former President of YouGov Peter Kellner was expounding his own theory that greater persistence in reaching respondents would have precisely the opposite effect - it would turn up hidden masses of socially liberal people who are more likely to vote Remain.

It was rather amusing to see the personal attack that YouGov's current management then made on Peter Kellner. I seem to recall that whenever I criticised Kellner during the indyref or questioned his motivations, it was cited as evidence that I was plainly an uncivilised cybernat lunatic! It's gratifying to see they've belatedly come around to my way of thinking...

Seriously, though, it's hard to escape the impression that YouGov were casting around for a reason why they as an online pollster might be right and the phone pollsters might be wrong. They homed in on educational attainment and it gave them the results they wanted, but you do have to wonder if they could have magically 'proved' the complete opposite - ie. that phone polls are more accurate - if they had lavished their attention on a different factor instead.

The bottom line is that the uncertainty continues, although two newly-released polls showing a Leave lead obviously has a significant impact on the Poll of Polls. (Both polls are somewhat out of date, but they were mostly conducted within the last three weeks, and therefore qualify for inclusion in the Poll of Polls.)

(The Poll of Polls takes account of all polls that were conducted at least partly within the last three weeks. The online average is based on seven polls - three from ICM, three from YouGov and one from TNS. The telephone average is based on five polls - one from ICM, one from YouGov, one from ORB, one from Ipsos-Mori and one from ComRes.)

Friday, May 20, 2016

The furious reaction of Dunc "don't call me Dunc" Hothersall is probably sufficient to tell you that Keiran Pedley may be on to something in suggesting that, because Labour's only hope of regaining power at UK level depends on a deal with the SNP, it would be better to conclude that deal in advance of a general election and then sell it to the English people. Nothing's inevitable, of course, as the bogus claims of "absolute certainty" about the result of this month's Holyrood election proves. I can just about imagine a scenario in which Labour might return to being competitive in Scotland by 2020 - it would depend on the constitutional issue fading from people's minds, and on Jeremy Corbyn being replaced by a charismatic leader who looks set to sweep the hated Tories from office. In those circumstances, the Scottish electorate's priorities might change. But the overwhelming likelihood is that one of three things will happen -

1) Corbyn will remain leader.

2) Corbyn will be replaced by someone of like mind, such as John McDonnell (who admittedly is more charismatic, but is still regarded as unelectable in England).

3) Corbyn will be replaced by a bland, uninspiring identikit centrist politician in the mould of Ed Miliband or Andy Burnham.

If that's the case, Pedley is surely right in saying that the game's a bogey for Labour in Scotland. But what intrigues me - and what Pedley doesn't address - is the electoral consequences of any deal. If we're talking about a formal pre-election Labour-SNP pact to keep the Tories out, how could the two parties justify putting up candidates against each other? Labour in particular would find it very difficult to pitch for votes in seats that are already held by the SNP - which is virtually every seat in Scotland. Logic would therefore seem to dictate that they should practically sit out the election north of the border.

Ironically, that would probably benefit only the Tories. The former Labour heartlands are highly likely to vote SNP anyway, and the absence of Labour candidates would simply lead to bigger majorities in seats that were never in doubt. But in heavily No-voting areas like the Borders and the rural north-east, the SNP could arguably do with Labour opposition to help split the unionist vote and keep the Tories out.

If an anti-Tory pact isn't going to prove counterproductive, the details are going to need some careful thought.

This week has been the perfect example of what drives me crazy about people's irrational responses to individual opinion polls. We've had seven EU referendum polls since Monday, and this is what they've shown -

* Five reported a Remain lead.

* Two reported a Leave lead.

* Four reported a swing to Remain.

* Three reported a swing to Leave.

As you can see, the overall picture is as clear as mud, and yet there have been big movements on the currency and betting markets, all apparently attributable to just one of those seven polls - the Ipsos-Mori telephone poll showing the Remain lead surging from ten points to eighteen. Now, to be fair, it's arguable that the trend shown by Ipsos-Mori is more important than the modest progress made by Leave in some of the other polls. If there isn't a dramatic indyref-style convergence between telephone and online polling as we approach referendum day, I would imagine the Leave campaign will be hoping to get to mid-June with a clear lead in online polls, and a deficit of no more than a few points in telephone polls. An eighteen-point lead for Remain in a new telephone poll obviously makes the latter aim seem less likely to be realised, so from that point of view Remain are probably happiest with the new batch of numbers. But there's still no excuse for completely discounting the more Leave-friendly part of the story.

One thing is for sure - online polling this week has entirely failed to corroborate the Ipsos-Mori trend. The ICM and TNS online polls both showed Leave in the lead and a small pro-Leave swing - which perhaps can be explained by normal sampling variation, but certainly can't be easily reconciled with a supposedly massive surge for Remain. The only other online poll was from YouGov, and showed the Remain lead increasing from 2% to 4% - but half of that small increase was an illusion caused by a methodological tweak. Essentially, the state of play was unchanged, which again contradicts Ipsos-Mori.

Even the other telephone polls have provided only limited backing for the Ipsos-Mori narrative. In line with YouGov, the ICM telephone poll on Monday showed a statistically insignificant increase in the Remain lead. The ORB phone poll did show a bigger increase, but as I've noted many times, the trend reported by ORB phone polling has from the start looked suspect, because it's been completely out of line with other data. The latest piece of the jigsaw arrived in the form of a new ComRes phone poll last night, which showed a slight improvement for Leave. That, I would suggest, tips the balance and makes it more likely than not that the Remain surge in the Ipsos-Mori poll was at least somewhat exaggerated.

ComRes have started using their turnout-adjusted model for their headline numbers, which makes a comparison with their previous poll more complicated. Here are two different sets of like-for-like comparisons -

Old methodology :

Remain 51% (n/c)
Leave 41% (+1)

New methodology :

Remain 52% (-1)
Leave 41% (+3)

The swing to Leave on both counts is small and isn't necessarily genuine, but it's unlikely the reported movement would be in that direction if Ipsos-Mori were correct about the Remain lead almost doubling.

So is there any particular reason why the Ipsos-Mori poll might have led us astray? Anthony Wells noticed a couple of striking things about the poll - that the percentage of Tory voters backing Remain was much higher than in other polls, and that a disproportionate amount of the boost for Remain came from mere 'leaners', ie. people who initially said they didn't know how they'd vote, but who opted for Remain when pressed by a follow-up question to say which way they'd be more inclined to vote. Ipsos-Mori include those people in the headline numbers for a good reason - past history shows it makes the results more accurate. But all the same, this referendum is an (almost) unprecedented event, and there's a good deal of uncertainty about the turnout. It could be that many of the Remain leaners found by Ipsos-Mori don't really care that much about the issue, and will be less likely to vote at all. A whopping 83% of the full sample gave a firm opinion on the main voting intention question, which is obviously much, much higher than the eventual turnout is likely to be - and further calls into question whether the others will ever make it to the polling station, regardless of their apparent inclination to favour Remain. But even if the leaners do vote in decent numbers, their opinions clearly look more susceptible to change.

* * *

SCOT GOES POP POLL OF POLLS50/50 ONLINE/TELEPHONE AVERAGE :Remain 47.8% (+2.0)Leave 41.7% (+0.4)ONLINE AVERAGE :Remain 43.3% (+0.7)Leave 44.0% (+0.5)TELEPHONE AVERAGE :Remain 52.3% (+3.3)Leave 39.3% (+0.3)(The Poll of Polls takes account of all polls that were conducted at least partly within the last three weeks. The online average is based on nine polls - four from ICM, two from YouGov, one from TNS, one from ORB and one from Opinium. The telephone average is based on four polls - one from ICM, one from ORB, one from Ipsos-Mori and one from ComRes.)

It's absolutely uncanny. In spite of the spectacular progress made by the Liberal Democrats two weeks ago, when they surged from a pathetic 5 seats last time around to a stunning 5 seats this time (prompting the cracking open of champagne and joyous hugs between Willie Rennie and Tim Farron), I somehow just had this weird feeling - call it a premonition - that Rennie's bid to become First Minister would all go horribly wrong and he would lose to the SNP's Nicola Sturgeon by a humiliating margin of 63 votes to 5. It seemed so improbable, and yet I should have trusted the nagging voice inside my head, because that's exactly what happened.

But give Rennie his due - at least he gave us an election. Ruth Davidson, by contrast, didn't make our day - she ran away. I couldn't help but smile at this rather charitable interpretation of her decision on the BBC website...

"Ms Davidson will not stand this time, after pledging during the election campaign to form a strong opposition."

That implied, somewhat amusingly, that the only reason Davidson wasn't putting herself forward as First Minister was because it would be a breach of a clear manifesto commitment that she wouldn't try to get elected. I felt sure that she wouldn't be mad enough to attempt that excuse herself - after all, you can't control how many people vote for you, so what would she have done if there had been a Conservative majority? Would she have instructed half her parliamentary party to resign immediately? Or would she have voted for Nicola Sturgeon just so that she could have the SNP government she promised to oppose? It gives a whole new meaning to the phrase "Vote Tory, get SNP".

But yes, bizarre though it may seem, Davidson confirmed in her speech that she wasn't trying to get elected because she had promised to be the opposition. The mind boggles.

* * *

EU referendum polls are like the proverbial London buses - you wait a whole week for one, and then four turn up at once. Since the two ICM polls were released yesterday afternoon, we've had another two from other firms...

Should the United Kingdom remain a member of the European Union or leave the European Union?

TNS poll (online) :

Remain 38% (-1)
Leave 41% (+5)

ORB poll (telephone) :

Remain 55% (+4)
Leave 40% (-3)

Remain activists were getting very excited about the ORB poll last night, but I'm not really sure what to make of it. The trend shown by ORB phone polls has been as mad as a bucket of frogs, and so far hasn't borne any resemblance to what phone polls from other firms have been reporting. When we have three polls from other firms conducted at around the same time (including a phone poll from ICM) all showing either a steady picture or slight progress for Leave, I think we have to be a bit sceptical about the ORB findings. It's not completely impossible that they're just ahead of the game in picking up a Remain surge earlier than other firms, but we'll just have to wait for more information to find out.

The other slightly irritating thing about ORB phone polls is that it's never entirely clear which set of figures should be treated as the headline figures. The Telegraph, who commission the polls, usually seem to give more emphasis to the turnout-filtered figures - which in this case are much more favourable for Leave (51% Remain, 45% Leave).

* * *

SCOT GOES POP POLL OF POLLS

50/50 ONLINE/TELEPHONE AVERAGE :

Remain 45.8% (+1.2)
Leave 41.3% (+0.1)

ONLINE AVERAGE :

Remain 42.6% (-0.5)
Leave 43.5% (-0.3)

TELEPHONE AVERAGE :

Remain 49.0% (+3.0)
Leave 39.0% (+0.5)

(The Poll of Polls takes account of all polls that were conducted at least partly within the last three weeks. The online average is based on eleven polls - four from ICM, two from YouGov, two from TNS, one from ORB, one from BMG and one from Opinium. The telephone average is based on three polls - one from ICM, one from ORB and one from Survation.)

Monday, May 16, 2016

After a whole week without a single EU referendum poll (almost unbelievable this deep into the campaign), ICM have broken the drought with simultaneous phone and online polls, which once again show the traditional disparity. It looks like public opinion hasn't budged much - the phone poll shows a statistically insignificant one-point increase in the Remain lead, and although the 4% Leave lead in the online poll is unusually high, it's still within the normal range when you take account of the fact that ICM made a couple of Leave-friendly tweaks to their online methodology a few weeks ago.

Should the United Kingdom remain a member of the European Union or leave the European Union?Online poll :Remain 43% (-1)Leave 47% (+1)Phone poll :Remain 47% (-1)Leave 39% (-2)

In line with the Kevin Williamson theory that opinion polls are always 100% accurate and that voters never, ever change their minds as an election approaches (as long as Boris Johnson isn't photographed with a copy of The Sun, naturally), we now know the result of the vote on June 23rd. Remain are going to win the phone referendum, and Leave are going to win the online referendum. It remains to be seen who will win the paper and pencil referendum, though, and come to think of it, that could be quite important.

Intriguingly, Martin Boon of ICM has parted company with his counterparts in ComRes, and also with Matt Singh of the Number Cruncher website, by disputing the assumption that the Remain-friendly phone polls are likely to prove more accurate. Matt Singh's reasoning on that point has always seemed a little suspect to me - for example, he prayed in aid the fact that online polls overestimated the UKIP vote in the May 5th elections (including the Scottish Parliament election), but it could well be that turnout patterns are going to be radically different next month. The EU referendum is the ultimate 'home fixture' for UKIP supporters, and it's not hard to see why they would be considerably less motivated to go out to vote in a Holyrood contest.

* * *

SCOT GOES POP POLL OF POLLS

50/50 ONLINE/TELEPHONE AVERAGE :

Remain 44.6% (n/c)
Leave 41.2% (+1.5)

ONLINE AVERAGE :

Remain 43.1% (+3.5)
Leave 43.8% (+4.7)

TELEPHONE AVERAGE :

Remain 46.0% (-3.5)
Leave 38.5% (-1.8)

(The Poll of Polls takes account of all polls that were conducted at least partly within the last three weeks. The online average is based on ten polls - four from ICM, two from YouGov, one from ORB, one from TNS, one from BMG and one from Opinium. The telephone average is based on two polls - one from ICM and one from Survation.)

Sunday, May 15, 2016

It's been suggested today that because the SNP won a clear mandate at the Holyrood election, it's wrong for the opposition parties to press for the repeal of the Offensive Behaviour at Football Act. It may surprise you to hear that I don't agree with that line of argument. In this particular case (and it's a very rare one), a tiny majority of MSPs with a vote - 65 out of 128, or 50.8% - were elected under the banner of a party that opposes one specific SNP policy. If those MSPs want to attempt to use that numerical edge to reverse the policy, there's no democratic problem there - quite the reverse, in fact. But the operative word is "attempt". There's a fair chance this episode may actually end up demonstrating just how strong the SNP's arithmetical position is. The opposition parties will need total discipline across four party groups who agree on little else and who don't particularly like each other - and that will only get them to 65 votes. By contrast, the SNP group is cohesive and should easily be able to turn out 63 votes. There's very little margin for error as far as the opposition are concerned, and the devil is often in the detail. At this stage, the smart money should probably be on the Act remaining in force.

However, if by any chance the repeal attempt succeeds, the Tories and Labour are quite right to say that we have to accept that there is a 65-63 mandate for that to happen, albeit a complicated four-party mandate. But the principle cuts both ways. The 69-59 mandate given to the two main pro-independence parties is of course considerably less complicated and much more emphatic (one might almost say "decisive"). I therefore look forward to the unionist parties displaying their customary logical consistency, and showing the same respect for democracy that they demand of us.

By the way, my popular Labour MSP namesake has made this mind-boggling observation on the subject -

"Labour will work [with] other parties in the Scottish Parliament to repeal the Football Act. The law has become a symbol of the SNP's arrogance in government. No other party supported the law but the SNP used their majority to bulldoze it through anyway."

Kelly-speak to English translation : The SNP used the overwhelming mandate they received at the 2011 election to outvote the losing parties. This is in line with what majority governments are typically expected to do. Presumably Kelly thinks the Attlee government shouldn't have used its 1945 mandate to establish the National Health Service, but should have been less "arrogant" and entered into a compromise with the Tories instead. But hey, a 10% discount on medical bills would be almost as good as a free NHS, right?

I knew that the sight of Australia racking up a huge Eurovision lead on jury votes, but losing it all on the public vote, reminded me of something. It took me a while to put my finger on it, but of course it was the SNP totally dominating the constituency vote in the Holyrood election, before the list vote didn't work out quite so well thanks to flippin' tactical...well, you get the picture. But at least Nicola Sturgeon still won the election by a country mile - there's no such solace for the Aussies tonight.

Normally when one country is 100 points ahead, and the host says "but everything can still change", it provokes little more than a weary and hollow laugh. In fact that was probably the case tonight, because I don't think anyone really thought Australia could be caught, but the new points system worked its magic better than we could have dreamed. It won't always be like this - often in the Melodifestivalen (where the system is borrowed from) the juries and public are in agreement. But a split decision is probably more likely to occur at the Eurovision, simply because the public vote is always heavily distorted by neighbourly bloc voting.

From the point of view of my own prediction, it was a slightly irritating final outcome, because I got so many things right, and yet I still somehow contrived to get the overall winner wrong. I correctly predicted that Australia would win the jury vote and would do reasonably well on the public vote. I was also correct that Russia would be hammered by the juries but would win the public vote. But what I didn't foresee was that a third country would come through the middle by finishing second on both the jury and public votes, and in particular I didn't foresee that the country that would do that was Ukraine. They were doing pretty well in the betting, and some Eurovision bloggers predicted they would win, but no matter how many times I listened to the song, I just couldn't see it happening. I thought it was too dark and complex to do well on the public vote (even allowing for Ukraine's in-built advantage courtesy of the ex-Soviet bloc vote), and I wasn't even 100% convinced that the juries would like it. I suppose part of the explanation is the emotion in the performance, although even that didn't come across as fully as it might have done if the song had been entirely in English.

But I can't say I'm disappointed - I would have preferred Australia, but the most important thing is that a credible, non-formulaic song won out over Russia's derivative effort. (And of course if Australia had won, the contest would have faced credibility problems of an entirely different sort.)

Graham Norton is a great commentator, but I do think he got a bit of deserved comeuppance tonight after repeatedly slating the Georgian song and saying it was baffling that it had made it through to the final when Ireland hadn't. (It really, really wasn't baffling at all.) He clearly wasn't sure quite how to react when the UK jury, comprised entirely of music professionals who presumably know their stuff, gave Georgia the maximum twelve points! Norton also isn't quite as sharp as Terry Wogan used to be in his observations on the voting. In this case, he was reading far too much into the lack of political voting among the juries, which actually isn't a great surprise if you look at the voting patterns prior to telephone voting being introduced in the late 1990s. Greece and Cyprus used to swap twelve points as a matter of routine, but that was pretty much it - there was no reliable Nordic bloc vote, or anything like that. There also shouldn't have been any surprise tonight that the UK did much less well on the public vote than with the juries - that's been a fairly consistent pattern in recent years, although until now it hasn't been quite so visible.

There used to be a tradition at the Olympics that the IOC president would finish his remarks in the closing ceremony by declaring the latest edition of the Games "the best ever". I think the EBU could be forgiven for making an equivalent boast tonight. I don't think the contest has ever been as well hosted, the jokes were actually funny rather than cringe-inducing (which is almost unheard of), the climax of the voting was a thriller (which has been rare of late), and the quality of the music was pretty high (by Eurovision standards, I mean, which is the only test that can be meaningfully applied). It really was the complete package - no complaints at all.