Friday, July 18, 2014

As I pointed out a few days ago that the rounding in the ICM poll had flattered No slightly, I should in fairness also point out that the reverse has happened in the new TNS-BMRB poll. For the whole sample, Yes have been rounded up from a position of Yes 43.6%, No 56.4%. Once again, though, a significant difference is made if you simply remove the 3% of the sample who say they will definitely not be casting a vote in September, which strikes me as an eminently sensible thing to do - that in itself is sufficient to take us up to Yes 44.3%, No 55.7%.

Among the 74% of the sample who say they are certain to vote, Yes have also been given a little boost in the published figures by the rounding - on the unrounded numbers it's Yes 44.6%, No 55.4%. However, for the second TNS poll in a row, the highest Yes percentage of all is found among the 85% of the sample who say they are either certain or very likely to vote - on that measure it's exactly Yes 45.0%, No 55.0%. That may be significant, because if I was going to hazard a wild guess as to what the turnout will be, I think I'd plump for something closer to 85% than 74%.

I couldn't resist a little peek at Political Betting last night, to see what inventive forms of denial the dark hordes were coming up with to explain away the apparent swing to Yes. I've identified a couple of particular favourites. First up is one from a certain Aberdeenshire Conservative activist...

"Deep breath, and relax. Scotland is on holiday."

This presumably implies that it's only affluent No-voting Scots who are on holiday, and that TNS are utterly incapable of coping with that problem by means of their weighting scheme. It's worth pointing out, though, that exactly the same Aberdeenshire Conservative activist once dismissed the results of a bad poll for the Tories on the grounds that "it's Christmas", even though it was in fact late November. By the same token, I'm guessing that any bad poll conducted during any part of spring is an irrelevance because "it's Easter", and as for January or February - forget it. Lovers are far too busy gazing into each other's eyes in anticipation of Valentine's Day.

And this was the other one -

"The Don't Knows are also Nos."

Hmmm. That would have been a much better line if this poll hadn't specifically asked Don't Knows which way they are leaning, and found that they are breaking slightly more for Yes. Adding in undecided leaners to the voting intention numbers has a small but significant effect across the board, although on some measures it would be disguised by rounding. All of the figures below exclude the hard-core of undecideds who are unable to give an answer even when pressed.

Should Scotland be an independent country?

Whole sample, includes undecided leaners :

Yes 44.3%

No 55.7%

Whole sample other than the 3% who will definitely not vote, includes undecided leaners :

Yes 44.8%

No 55.2%

Respondents who are certain or very likely to vote (equivalent to 85% turnout), includes undecided leaners :

Yes 45.3%

No 54.7%

Respondents who are certain to vote (equivalent to 74% turnout), includes undecided leaners :

Yes 45.3%

No 54.7%

One question mark that John Curtice has raised about this poll is that Yes have been weighted up from their position in the raw data by slightly more than usual. That seems to be largely because TNS have had to make a bigger adjustment for past vote recall, due to there being more people in the sample who say they voted Labour in 2011 than who say they voted SNP. In order to conclude that the weighting procedure is misguided, though, you'd have to believe that there is a significant problem of people being unable to accurately recall how they voted in 2011, which would flatly contradict evidence from other pollsters that recall of 2011 vote is much more accurate than recall of 2010 vote. In any case, even if we assume for the sake of argument that people who recall voting Labour are being downweighted too much, that isn't the whole story by any means - TNS also dramatically upweight people who say they didn't vote in 2011, or who can't remember how they voted. Both of those groups are more No-friendly that the overall sample. So it's not immediately clear that the position would be that much worse for Yes if weighting by past vote recall was eliminated altogether. SNP voters haven't actually been upweighted much at all - only from 224 to 228, which has a negligible effect on the results. As usual, a big part of the reason for the disparity between the raw data and the published results is that the heavily No-friendly oldest age group has had to be downweighted significantly to bring them into line with their real population share.

Thursday, July 17, 2014

TNS-BMRB have on average been the third most No-friendly out of the six BPC-affiliated firms. Their last poll showed Yes on a new high watermark for the campaign - but even that was only 41.4% with Don't Knows excluded, as compared to the highs reported by ICM, Panelbase and Survation of between 47.1% and 48.3%. It was also only fractionally higher than the figure that Yes had been hovering around in the firm's polls for several months. However, just when we were beginning to wonder if Yes were ever going to make a telling breakthrough with TNS, it's finally happened tonight - and in quite some style, with the No lead slumping by 6% after Don't Knows are excluded.

Should Scotland be an independent country?

Yes 44% (+3)No 56% (-3) With Don't Knows left in, these are the figures -

Yes 32% (n/c)No 41% (-5)

On the latter measure, Yes remain on the new high watermark for the campaign that they reached in the previous TNS poll - before that poll, they'd never been higher than 30%, or at least not since TNS introduced a major methodological change. By complete contrast, the No campaign's 4% increase in the last poll (which was partly an illusion caused by rounding) has been more than wiped out, and they're now back down to a vote share that equals their all-time low. Although on face value this looks like a straight swing from No to Don't Know, it's more likely that the last poll was simply an outlier in showing a big decrease in the undecideds. That would mean there has been genuine movement from No to Yes at some point over the last few weeks.

As you'll doubtless have spotted already, the headline No lead is just 9% - the first time it has hit single figures with TNS in the campaign so far. That's a 5% drop on the last TNS poll, and is 3% lower than the previous record low shown by the firm.

I've pointed out before that swings to Yes reported by Ipsos-Mori and TNS are considerably more important than swings reported by other firms, for the simple reason that they're the only two active referendum pollsters that actually seek out a fresh 'real world' sample in every poll, rather than rely on volunteer online panels. Until now, both have tended to be firmly on the No-friendly end of the spectrum. That isn't to say they're necessarily more accurate than others, because there are plenty of advantages to the online approach as well - it's more anonymous, for starters, and respondents are more likely to give honest answers. But it's nevertheless highly encouraging to see an old-fashioned face-to-face pollster like TNS produce the kind of numbers that have previously only been reported by online firms. Indeed, as things stand TNS are showing a slightly higher Yes vote than ICM (after Don't Knows are excluded), which is quite a turn-up for the books.

On the other hand, the perennial problem with TNS is that there's always quite a long gap between fieldwork and publication, meaning that their figures are slightly out-of-date by the time that we see them. In this case, that will complicate any attempts to use this new poll to resolve the mystery of the contradictory trends we've been seeing recently from different firms. The TNS fieldwork took place between the 25th of June and the 9th of July, meaning that it had the same start-date as the most recent YouGov poll, but a much later end-date. So it certainly adds to the weight of evidence that the slight drift towards No reported by YouGov recently has probably just been margin of error "noise", and it leaves open the possibility that there has been movement to Yes since YouGov's fieldwork concluded. The fieldwork for last week's Survation poll overlapped with the tail-end of the TNS dates, which is useful, because both firms are in agreement that Yes have reached a new high. However, the most up-to-date poll remains the ICM poll published at the weekend, which finished a couple of days after TNS had stopped interviewing. Confusingly, that poll showed a small increase in the No lead - but the changes were consistent with margin of error noise, and the headline figures remained well within ICM's normal range.

So although the TNS findings dramatically increase the chances that there has been real and substantial movement to Yes recently, they still don't constitute absolute proof. As ever, we'll just have to await the next poll for more information. One thing we can certainly say, though, is that there's no credible evidence at all of a swing against Yes - shouldn't that be happening by now, if the predictions made last year by certain "experts" (and Ian Dunt) have any validity?

* * *

UPDATE : Even better news - the STV website is reporting that the TNS figures for respondents who say they are certain to vote are as follows...

Yes 37% (+2)No 46% (-3) No word yet on the figures with Don't Knows excluded, but a rough calculation suggests that they're most likely to be Yes 45% (+4), No 55% (-4). That would certainly explain why Yes Scotland tweeted that this poll showed Yes at 45%, rather than 44%!

* * *

REQUIRED SWINGS Although I'm going to persevere with the Poll of Polls, I'm coming round to the idea that it may not be the best way of summarising the state of play, given the huge ongoing disparity between the figures produced by different firms - it's just providing an "imaginary middle". So I'm going to start giving the swings that would be required for a certain number of pollsters to either show Yes in the lead, or a dead-heat. For simplicity, these are based on the rounded numbers used for publication, and with Don't Knows not excluded. (It wouldn't be possible to use unrounded numbers across the board anyway, because the notoriously secretive firm YouGov never reveal their unrounded results.)

Swing required for 1 out of 6 pollsters to show Yes in the lead or level : 1.5%Swing required for 2 out of 6 pollsters to show Yes in the lead or level : 2.5%Swing required for 3 out of 6 pollsters to show Yes in the lead or level : 4.5%Swing required for 4 out of 6 pollsters to show Yes in the lead or level : 5.5%* * *SCOT GOES POP POLL OF POLLS
This update of the Poll of Polls essentially reverses the small boost for No that was seen last time round - the No lead with Don't Knows taken into account is now back below 11%.

(The Poll of Polls is based on a rolling average of the most recent poll from each of the pollsters that have been active in the referendum campaign since September 2013, and that adhere to British Polling Council rules. At present, there are six - YouGov, TNS-BMRB, Survation, Panelbase, Ipsos-Mori and ICM. Whenever a new poll is published, it replaces the last poll from the same company in the sample. Changes in the Poll of Polls are generally glacial in nature due to the fact that only a small portion of the sample is updated each time.)

And here are the long-term trend figures, with updates prior to Easter recalculated to remove the inactive pollster Angus Reid ...

The No campaign's lead in the Poll of Polls mean average (not excluding Don't Knows) :

Wednesday, July 16, 2014

There are already doctored photos doing the rounds on Twitter of the Z-list celebrities from the much-mocked 'Let's Stay Together' video, with their patronising "We love you, Scotland (you're such a fab nuclear weapons base)!!!!" messages being replaced with "Yes" or "Aye". But the photo of Eddie Izzard below doesn't need any doctoring - it really does show him campaigning with Elaine C Smith under a 'Scotland Says Yes!' banner.

That was just three years ago, when both were campaigning for a Yes vote in the AV referendum. It seems that Izzard's zeal for constitutional reform has very rapidly deserted him, because he now wants Scotland to reject a proportional voting system for national elections, a written constitution, the abolition of unelected legislators, and all of the other long overdue steps forward that would happen with independence, and that would only happen with independence.

Alternatively, it could just be that he (along with Tony Robinson, Richard Wilson and Ross Kemp) is a blindly loyal Labour man, and just goes along with whatever stance the Labour leadership take on any given issue. If so, it's rather sad.

As ever, four little words : True Love Isn't Possessive.

* * *

A couple of interesting straws in the wind, albeit based on extremely small sample sizes - the Scottish subsamples from the new GB-wide telephone polls conducted by ICM and Ipsos-Mori both show the SNP in the lead in Westminster voting intentions...

You've probably heard by now that the unspoofable Tory MP Rory Stewart is using crowdfunding in an attempt to finance his new "Stones Against Self-Determination" project (aka "Plan C"). The highlight of the fundraising page is a photo bearing the caption "the first stages of design", but which actually appears to be of Rory and a couple of groupies randomly performing Tai Chi in a field. He's looking for £50,000, which I would imagine will be quite a challenging target, given that so few of his friends in the Conservative parliamentary party have any spare cash. But in a valiant and self-sacrificing effort to hurry things along, he's offering this irresistible perk to anyone prepared to stump up £1000 -

"£1000 - A walk with Rory Stewart, MPA tour of The Auld Acquaintance [ie. the pile of stones] followed by a walk in the beautiful Scottish borders with Rory Stewart OBE MP. Rory is an acclaimed author and has walked 6000 miles on foot across Iran, Afghanistan, Pakistan, India and Nepal. More recently, he walked 400 miles across his constituency of Penrith and The Border."
Three hours ago, the first £1000 donation came in from an anonymous benefactor - but he/she decided against claiming the reward of a walk with Rory. It does give a whole new meaning to the phrase "you can't even give it away".

* * *

PS. How exactly do you give someone a "tour" of a pile of stones? Isn't that another way of inviting them to walk in a circle for approximately twenty seconds?

* * *

PPS. Are you as impressed as I am that Rory didn't merely walk 6000 miles, but did it "on foot"?

Tuesday, July 15, 2014

I know we all love our anecdotes about finding Yes voters in unexpected places, so I thought you might enjoy this one. For the last four years or so, I've been going to the monthly meetings of a book club - in fact, I think I may have been on my way there when I suffered my excruciating pain/Wendy Alexander incidenta couple of years ago. It's exactly how you'd picture a book club - predominantly female and middle-class. It also tends to attract newcomers to the area who are looking for something to do and to meet new people, so there's always a significant minority of people from south of the border, and a few from further afield - before I even joined, a Korean woman who I was briefly friends with went to a couple of meetings, and since then there have been people from Denmark, Ireland, Germany, Russia, India, Australia and New Zealand.

There's a nomination-followed-by-vote system to decide which books to read each month, and at some point last year I found myself casting a decisive vote in favour of Dominion by C. J. Samson. I'm vaguely interested in history, so the idea of a speculative alternative history of the Second World War's aftermath appealed to me - but, alas, I was blissfully unaware that a large chunk of the book is made up of deeply offensive "the SNP would have collaborated with the Nazis" propaganda, and that Samson is one of the No campaign's fabulously wealthy donors. I was intensely irritated with myself when I found out, if only because my vote had indirectly led to Samson selling another 20-odd copies of his book, thus subsidising a tiny bit of his bloody donation!

Luckily, though, I had a chance to cancel out my mistake at the end of last month's meeting, because someone nominated A People's History of Scotland, written by Yes supporter Chris Bambery. So I voted for it, and thankfully that was again a vote that swung the balance. The meeting to discuss the book was earlier this evening. I went along knowing there would probably be a debate about the referendum, and also fully expecting that I would find myself in a tiny minority as a Yes voter. But I couldn't have been more wrong - person after person identified themselves as favouring Yes, including one who turned out to be an active campaigner. There was one man who spoke from an Old Left viewpoint, and who kept using the dread word "separation", but even he concluded by saying that his views had moved on and that, at the very least, he wouldn't be voting No. It got to the stage where it was so one-sided that the assistant organiser appealed for a contribution from anyone who is voting No, and at that point a couple of people who had been keeping quiet made the case against independence (doubtless Martin Boon would interpret that as compelling evidence of Shy No Syndrome!). But at the end, a quick vote was held, and there was a clear majority for Yes.

Colour me astonished. I'm inclined to say that if we can win among a demographic like that, we can win absolutely anywhere. But I know life is rarely that simple...

Monday, July 14, 2014

It's uncanny how often this has happened with ICM - to be fair it's just coincidence/bad luck, but the rounding has yet again flattered No in the published results of the new poll for Scotland on Sunday. On the headline figures that take account of Don't Knows, the true No lead before rounding is much closer to 10% than to the 11% we saw in the published numbers. And when Don't Knows are excluded, Yes are at 43.44%, and are therefore just the tiniest smidgeon away from being rounded up to 44% rather than down to 43%. Here are what the poll results look like when rounded only to one decimal place -

So the changes from last month (which the barking mad Huffington Post described as a Yes "nosedive") are even smaller than they appeared to be in the published results, which increases the overwhelming likelihood that they're just meaningless margin of error 'noise'.

You might also remember that after revealing the results of the ICM poll from two months ago, Kenny Farquharson made a big issue of the fact that if the firm's methodology hadn't just changed to take account of differential turnout, then Yes would have slipped to 40% (actually 40.4% on the unrounded numbers) rather than to 42.4%. Well, If you scour your memory of the last 36 hours or so, you'll probably realise that no-one has made that point this time, and for a very good reason - in this case the Yes vote before turnout weighting is applied is the same (43%) as in the published results. So whichever way you look at it, the No lead is clearly lower in this poll than it was two months ago - and by quite a substantial amount on one measure.

I'm slightly baffled as to how the Scotland on Sunday and John Curtice felt able to make a song and dance about how this poll supposedly shows that people are starting to trust the anti-independence parties on the delivery of new powers to the Scottish Parliament in the event of a No vote. In reality, the number of people who think no new powers will be delivered or that existing powers will be taken away is, at 45%, higher than the number of people who expect new powers to arrive (40%). It's possible that the electorate may eventually have the wool pulled over their eyes by the No campaign, but it certainly hasn't happened yet.

Now let's have a look at what John Curtice didn't tell us about the question asking people to rate themselves on a 1-10 scale of support for independence. The first thing to say is that this only turns up deep, deep into the question sequence, which is less than satisfactory, because by that point many respondents will be self-consciously trying to keep their responses consistent with what they've already said. Nevertheless, the results are much better for Yes than we've been led to believe. As a number of people confirmed on the previous thread, it's standard during canvassing to regard a '5' as being exactly in the middle (ie. a complete Don't Know), meaning that anything between 1 and 4 is on the anti-independence end of the scale, and anything between 6 and 10 is on the pro-independence end. Pedants and statisticians may bristle at that notion, because it means there are slightly more pro-independence numbers available, but nevertheless that's how it works - the human brain is programmed to see 5 as the mid-point in a scale that goes up to 10. When looked at that way, these are the results -

(The above numbers exclude the small number of respondents who refused to place themselves on the scale.)

Now are you starting to see why the Yes campaign are so bullish about the numbers that are being produced by that approach? The other point Stephen Noon made the other day (and unfortunately the ICM poll can't assist us with this one way or the other) is that, over time, people have been gradually moving up the scale towards independence, and that was happening even before any movement towards Yes was detectable in the published polls. If that's true, it could be hugely significant.

Of course it also means that the people who currently rate themselves as a 5 are particularly important, as they literally hold the balance. So what do they look like? Well, 40% of them didn't even vote in the 2011 election, so that limits the numbers of clues available. 22% voted Labour, and 12.5% for the SNP. Most say they are undecided on how they will vote in the referendum, but a significant minority of 24% currently say they will vote No, which goes a long way towards explaining the disparity with the headline numbers - these are the soft Nos we keep hearing about.

There's also a weird amount of "cross-voting" going on - supposed No voters who place themselves on the pro-independence end of the scale, and supposed Yes voters who place themselves on the anti-independence end of the scale. Were these people just confused by the question? It's likely that at least some of them were, given that 1% of No voters rate themselves as a 10, ie. "completely for an independent Scotland"! But the 5% of No voters who rate themselves as a 6, 7, 8 or 9 are potentially more significant - those could well be soft Nos (very, very soft Nos). By contrast, just 1% of Yes voters rate themselves as a 2, 3 or 4.

Annoyingly, ICM haven't asked a country of birth question this time (or if they have there's no sign of the results). But on past form it seems reasonable to assume that English-born people are once again heavily over-represented in the sample, which almost certainly means that the headline Yes vote is a bit too low, and the headline No vote is a bit too high. However, women aren't quite as over-represented as they were in the previous two polls, which is a touch irritating given that (very unusually) more women than men in this poll say they will vote Yes!

Sunday, July 13, 2014

Just a very quick post, because I'm almost lost for words. This is the way Huffington Post UK have chosen to headline an ICM referendum poll which shows a mere 2% decrease in the Yes vote since last month (a change which could easily be, and probably is, an illusion caused by the standard 3% margin of error), and which also shows a lower No lead than just two months ago...

"Support For Scottish Independence Nosedives, Poll Shows"

How is anyone going to be able to take a word they say on the referendum seriously in future?

This month's ICM poll has just been released, and shows only margin of error changes. The headline numbers are well within the firm's normal range, which is a No lead of between 3 and 12 points -Should Scotland be an independent country?

Yes 34% (-2)No 45% (+2) With Don't Knows excluded, it works out as...

Yes 43% (-2)No 57% (+2) On both measures, the No lead remains lower than it was two months ago, when ICM were showing a 12-point gap if Don't Knows were taken into account, and a 42/58 split if they were stripped out.

I must say this is all rather unexpected, given that a veritable Rogue's Gallery of (exclusively male) No campaign staffers were lining up on Twitter just an hour or two ago to gloat about what the poll would shortly show. Although that form of gloating is not untypical, I'd never seen it on quite that scale before, so I started to assume they might have something out of the ordinary to crow about. But it turned out to be yet another classic case of the unspeakable in pursuit of the unspinnable.

So where does this leave the overall state of play? We're back to "as clear as mud" once again. Let's take the six BPC pollsters in turn -

ICM : The most volatile of the six firms are showing a No lead that is within their very wide normal range, but which is towards the upper end of it. It remains to be seen whether they've tinkered with their methodology yet again in this poll - hopefully they won't have been so foolish as to introduce an artificial 'Shy No' adjustment, given that there is literally zero evidence for the existence of 'Shy No Syndrome'.

Panelbase : Showing an all-time high for Yes of 48.2%, but haven't reported for a few weeks.

Ipsos-Mori : Also showing an all-time high for Yes, but haven't reported for a few weeks and on past form are unlikely to surface again for quite a while.

YouGov : This most obsessively secretive of the six firms, notorious for adjusting the Yes vote downwards with their "Kellner Correction", have shown Yes slipping back slightly from an all-time high recorded a couple of months ago - although those changes are perfectly consistent with margin of error 'noise'.

Survation : Showing an all-time high for Yes of 47.1%, and are bang up to date.

TNS-BMRB : Showing Yes at a high watermark for the campaign of 41.4%, although that's only fractionally above the figure that Yes have been hovering at for a few months.

So the overall picture is consistent with Yes making gains or with a static position - or even with a modest recovery in the No lead, although that seems somewhat less likely. All we can really say is that it will take at least a couple more polls before things start to make sense again.

You may have heard Yes Scotland's chief strategist Stephen Noon on the Derek Bateman show the other day, making clear that he is confident of victory based on a more sophisticated form of polling and canvassing than we normally see in published polls. But in his Scotland on Sunday analysis of tonight's new poll, Professor John Curtice (who appears to be claiming part-authorship of the questions asked) suggests that the Yes Scotland approach has been replicated by ICM, and has failed to produce quite such an optimistic picture for Yes. His explanation is that when asked to place their support for independence on a scale from 1 to 10, no fewer than 7% of respondents plump for a '5', which is on the lower end of the scale - and Yes would need every single one of those people just to get up to a 49% share of the vote. But it seems to me there's a huge logical flaw in Curtice's thinking, and I'll be interested to see if Stephen Noon makes any comment on it. 5 may technically be on the lower end of a 1-10 scale, but the reality is that most people who want to place themselves exactly in the middle of the scale are likely to call themselves a 5 rather than a 6. That's just basic psychology - you think to yourself "what's half of 10?" So Curtice's conclusion that two-thirds of undecided respondents in this poll have consciously placed themselves on the anti-independence half of the scale should also be treated with extreme scepticism.

* * *

SCOT GOES POP POLL OF POLLS

As usual, I'll give a full-scale Poll of Polls update below, but given the ongoing disparity between various pollsters, I think it may be time to start taking a 'top and tail' approach as well. It is, after all, highly unlikely that the referendum result will magically turn out to be an exact middle point between the Yes-friendly and No-friendly pollsters - it's far more probable that one set of pollsters will be closer to the truth than the other. As of tonight, the 'top' (Yes-friendly) three pollsters are showing an average of exactly Yes 46%, No 54% - well within touching distance of victory for Yes with more than two months still to go.

Anyway, here is the more conventional Poll of Polls...

MEAN AVERAGE (excluding Don't Knows) :

Yes 43.1% (-0.5)
No 56.9% (+0.5)

MEAN AVERAGE (not excluding Don't Knows) :

Yes 36.8% (-0.4)
No 48.5% (+0.3)

MEDIAN AVERAGE (excluding Don't Knows) :

Yes 42.1% (-1.2)
No 57.9% (+1.2)

(The Poll of Polls is based on a rolling average of the most recent poll from each of the pollsters that have been active in the referendum campaign since September 2013, and that adhere to British Polling Council rules. At present, there are six - YouGov, TNS-BMRB, Survation, Panelbase, Ipsos-Mori and ICM. Whenever a new poll is published, it replaces the last poll from the same company in the sample. Changes in the Poll of Polls are generally glacial in nature due to the fact that only a small portion of the sample is updated each time.)

And here are the long-term trend figures, with updates prior to Easter recalculated to remove the inactive pollster Angus Reid ...

The No campaign's lead in the Poll of Polls mean average (not excluding Don't Knows) :