Wednesday, July 25, 2018

As I've spent a fair bit of the last 48 hours having exchanges (sometimes downright surreal exchanges) about this subject on Twitter, I thought I might as well make the point here as well. It's a very simple one. Here is how the Britain Elects account reported the results of the new YouGov poll a couple of days ago...

I'm not having a go at Britain Elects specifically, because the above is absolutely typical of how most news/political outlets summarise such polls - ie. with no sign of the SNP (or indeed of Plaid Cymru).

How do you think any reasonable person would be most likely to interpret the absence of the SNP? I'd suggest they'd reach one of two conclusions. Either: a) respondents in the poll were not given the option of expressing a voting intention for the SNP, or b) the SNP were on less than the 2% of the vote enjoyed by the Greens, the lowest-placed of the five parties that were deemed worthy of a mention in the summary. But both of those conclusions would be completely incorrect, meaning that by either accident or design people are being very seriously misled. In reality, the SNP and Plaid Cymru received 5% of the vote in this poll, putting them in a clear fifth place ahead of the Greens, and only just behind UKIP in fourth place. (Because YouGov lump the SNP and Plaid together as a single option for GB-wide polls, it's impossible to separate out the support for each of the two parties, but given what we know about their respective levels of support it's inconceivable that the SNP would have received less than 4% if offered as an option in their own right, and would still have been well clear of the Greens.) Why, then, is the sixth most popular party reported as if it was the fifth most popular? Why is the fifth most popular not even mentioned at all?

A mistake?

An oversight?

Nope, it's the intentional withholding of information, and it's done as a matter of routine. Over the last two days, apologists for this downright weird practice have put forward a number of speculative justifications for it, and not one of them makes any sense. I'll go through them individually.

"Not editing out the SNP's vote would give a misleading impression of the trend in Scotland, because trivial changes that might barely register at Britain-wide level would be enough to make a big difference in terms of seats." This doesn't stack up, because essentially the same is true of both the Greens and UKIP - any seats that they might win depend on very localised contests, meaning that their national share of the vote is hardly even relevant. In 2015, UKIP took 13% of the vote but won just a single seat. If the media can 'take the risk' of revealing information about the popularity of the Greens and UKIP that has little or no relevance in terms of seats, it's murderously hard to understand why the public must be 'protected' from similar information about the SNP. The bottom line is that in a first-past-the-post election, the number of seats won by each party is only very weakly correlated to the share of the vote. The winner of the popular vote may or may not be the largest party in terms of seats. A third party with 17% of the vote may win more than twice as many seats as it did a decade earlier with 23%. The purpose of polls is not first and foremost to predict the number of seats for each party, but rather to estimate each party's absolute popularity in terms of votes. In that respect, the fact that the SNP is on 5% of the vote in this YouGov poll is no more or less important than the fact that the Greens are on 2% or that UKIP are on 6%.

"The estimated vote for the SNP is less reliable than the vote for Britain-wide parties, because it is drawn from a tiny subsample, not the full-scale GB sample." Not true. YouGov allow respondents across Britain to select the SNP/Plaid as a voting intention option, as can be seen from the fact that the two parties between them have 1% support in London in this particular poll.

"Nevertheless, in practice the vast bulk of support for the SNP and Plaid comes from Scotland and Wales, so effectively is based on a subsample that is too small to be statistically reliable." That's really an argument for not taking individual subsamples too seriously, which indeed they shouldn't be. But the SNP's GB-wide vote is not a subsample figure - it's rounded to the nearest percentage point and therefore normally falls in a range between 3% and 5%. If anything, the SNP's reported vote is more stable than the reported vote for the Britain-wide parties and isn't subject to random variations outside the standard margin of error - which is what you'd expect if the charge of an unusual level of statistical unreliability had any truth to it.

"The SNP's support is not only effectively drawn from a small Scottish subsample, but one that might be incorrectly structured - for example, it might have far too many pensioners, or too many women." Not so. YouGov indicated a couple of years ago that they had decided to start structuring and weighting their Scottish subsamples separately to improve the accuracy of their polls. It seems highly unlikely that they reversed that decision at any point, because their subsample figures have become (relatively) more stable since then.

"The SNP should be edited out of poll results because not everyone in Britain can vote for them." That's a British nationalist argument rather than a statistical one, but it doesn't even make sense on its own terms, because not everyone in Britain can vote for UKIP or the Greens either. In the 2017 general election, the Greens stood in only 467 of the 650 constituencies, and UKIP stood in only 378 of 650. Both figures were sharply down on the candidates for each party in the 2015 election. Nobody has a clue how many candidates UKIP and the Greens will put up at the next election, which means that in all probability many respondents will have told YouGov in good faith that they plan to vote for one party or another even though they will not be able to do so. If reporting the SNP's Britain-wide vote "lacks context", reporting the Green or UKIP vote must inevitably lack a great deal more context. And yet nobody would dream of withholding that information (unless of course the numbers fell to a statistically insignificant level).

There is no possible logic to the exclusion of the SNP from poll summaries. It's an arbitrary decision rooted in Anglocentricity.

* * *

Fundraiser: If you find Scot Goes Pop's polling coverage useful and would like to help it continue, donations can be made via the 2017 fundraiser page. The initial £7000 target was reached last summer, but one year on that money has all been used up. I know there are always lots of very worthy pro-independence causes looking for support, so I've held off for as long as I possibly could before actively seeking donations again.

24 comments:

What were you expecting? The tactic is to drive down the SNP vote by pretending that they can't win/don't exist. Just like the good old days, pre-devolution.

In other news how can the water off Portobello beach be f*cking freezing after the hottest June and July in recorded history? Also how can the daily rectum possibly get away with claiming a staged photo as showing the bedraggled Nazi fresh from her ankle deep paddle? A clear case of Trumpish news for David Leask to investigate

I agree, but I think the real problem is their insistence that GB-wide polls are still a good idea. Although most parties are shared between England and Scotland, they often don't fall and rise at the same time. They're already excluding NI - why not exclude Scotland from these polls, too?

A new easy to use tactic has been used with some success for some time , instead of attacking the vile separatists ignore them , or as Thatcher said deprive them of the Oxygen of publicity they need so we had the farcical use of actors saying what the IRA would have said if they had been allowed to say it , i kid you not thats what happened , if it wasnt so far fetched the countries political system would make a good comedy film , or for some a real horror show .

What daes ma heid in is when the Scottish papers just copy these polls wi nae explanation of the absence of the SNP.That's why the walk-out was so effective. It was impossible to ignore.The Inverness indy rally will keep the pot boiling. Pity there are no elections as we'd be creating our own publicity.

As some of you know, I stumbled upon this nifty sure years ago doing a polling study on why polls were becoming less accurate. As games has pointed out a hundred times now??? These excuses are completely invalid.there is literally no way anyone with actual expertise in polling can believe these excuses.there is also a thing called subset over polling.in the old days, these were done maybe 20% of the time to check the overall poll. Of a qualified pollster actually thinks a sub sample may be inaccurate it INVALiDATES THE ENTIRE POLL! For example, you cannot poll in the upper Midwest on bowling night, parents on back to school night, news on Friday night, Catholics on Sunday morning, you can't poll Liverpool on a Saturday that Liverpool is playing Everton. If you do, you go back and make two sets of calls, one to previous set and one new random. Only if these match do you release. Does anybody seriously think if they are low on responses in the London suburbs they would release a poll showing SNP at 10%? No..they would tell the callers to keep calling until that " bucket " is full. This continued failure of pollsters to actually do their job is very frustrating.has anybody run these thru a news filter random study to see if they are just tweaking the results every few weeks to show change and not actually calling people? Two big USA pollsters did this for years before being caught.

Hey there, you never did answer my question about what stable end-point you see for "the devolution journey" that isn't independence or subsumption into England.

As for why it's anglocentric to leave out the biggest Scottish party, but include parties that are smaller than them, well, that's kinda obvious when I put it that way, isn't it?

And if you were serious about not including parties that "could form a government", then you're ignoring the possibility of coalitions. Which is just dumb (and should be arguing for the DUP to be included). And you'd also be arguing for the exclusion of everyone but the red and blue tory parties.

Sorry, what? On what planet could UKIP or even the Lib Dems "form a government"?

And you may not be aware of this, but the Tories and UKIP are the only parties that stand in all four "British countries", so if that's the rule, poll summaries are going to look a bit bare from now on.

After all the brouhaha in recent years about inaccurate polling, with the SNP being the third largest and most reliably unanimous voting bloc among the top three parties at Westminster, with the current Westminster regime dependent on the DUP to stay in power - with the DUP quite reliably voting with the Tories whether they are in open coalition or not - there can be no logical or statistical reason to not pay proper attention to polling the non-English parts of this precious, precious Union.

The most likely explanation is that pressure is being exerted on the pollsters to provide results that are inaccurate: at best pretty much meaningless, and at worst deliberately misleading: poll results are deliberately pauchled only for (party) political advantage and for purposes of propaganda, and, as is well known, in the worst of cases that extends to the results of elections themselves.

Are pollsters really so incompetent that they fail to include 16- and 17-year-olds in Scotland in their sampling in Scotland for independence referendum voting intentions (maybe 15-year-olds as well who will likely be able to vote by the time the referendum comes round), so incompetent that they insist on lumping together voting intentions for the non-Unionist parties in Scotland and Wales, i.e., outwith England? Do they act out of concious or unconscious anglocentricity, or do they do it to deliberately downplay, trivialize and suppress the true extent of the resistance to the Tories - and the other Unionist parties - outwith England?

For once I'm with the tinfoil-hatted, swivel-eyed brigade of conspiracy nuts: we live in an age in which the legitimacy of democratic elections is routinely and deliberately being undermined by various actors using an increasingly wide range of means.

Against that we can posit the attribution of blame for the bias within the whole BritNat meeja machine: are they doing it deliberately in the full awareness of what they are doing, are they under pressure of any kind to do it, or are they doing it out of unconscious bias based on - oh, collective narcissism, anglocentricity, jingoism, a form of xenophobia, a superiority complex - you can call it various things. A rose smells just as sweet under any other name, after all.

From what you say, Charles, we can deduce that those who commission the polls, and doubtless quite a few others, actually know what the real figures are.

That goes to show that the figures are being actively withheld from the public, even - especially - in the parts of the UK where they would be of greatest interest and significance.

In the light of the constant barrage of negative propaganda or censorship-by-omission that we get here from the Great BritNat meeja machine, I am going to put my tinfoil hat on and say that leaving the figures for Plaid and the SNP to be swamped by the UK figures, or even omitted entirely, is simply a reflection of that overall meeja practice. I myself have noticed an increasing trend toward leaving them out completely.

There is, in a backhanded sort of a way, some comfort to be derived from that: it means that the opposition are running scared, because you can bet your bottom dollar that if the polling results were bad for the SNP, they'd be all over it like a rash.

We know the principle: for the BritNat meeja, and the BritNat Establishment, there can be no good news about the SNP; even good news has to be spun as SNPBAAAAAAD; and if they can't get away with that, then instead of negative propaganda there's always censorship by omission.

What it tells me is that Britain Elects and the rest of them have nothing to say to Scottish politics. It really should be England Elects.

We know the SNP are around 42% in the full Scottish polls so 3.5% to 4% would indicate that the sub samples are sound. They used to report these numbers so I am inclined to the view that they have been told to stop publicising them. No recent election has suggested that the aggregate of sub samples for Scotland were any more inaccurate than the rest of the polls.