Monday, October 21, 2013

By-elections called, by-elections polled

With a keen sense of timing, Forum Research has released new poll results via Global on the four federal by-elections that were called yesterday for Nov. 25. The results are strikingly good for the Liberals, with leads in three of the four by-elections (two of them rather comfortable). But it may not be as clear cut as that. Let's look at the polls, the potential issues, and what to take from them.

The four polls show strong numbers for the Liberals, with an average of 40% support in ridings they averaged 24% in the 2011 federal election. That 16-point gain actually matches rather well with the party's increase nationwide since the last vote. The Conservatives were, on average, down about 12 points while the NDP was down about nine points.

In Bourassa, Emmanuel Dubourg led with 47% support against 18% for Stéphane Moraille of the NDP. The Bloc Québécois followed with 15% and the Greens with 12%. However, Forum listed Daniel Mailhot and Georges Laraque as the candidates for the Bloc and Greens. The actual Bloc candidate is Daniel Duranleau (Mailhot ran in 2011), though I don't suspect that would make much of a difference. But Laraque is no longer the Green candidate, and that absolutely would make a difference. If Laraque's vote is cut in half or more, where would it go? Moraille seems like the obvious destination.

The numbers in Brandon-Souris were the most surprising of the set, with Rolf Dinsdale of the Liberals leading with 39% to 35% for Larry Maguire of the Conservatives. The NDP and Greens were tied at 12% apiece. Considering the riding was easily won by the Tories in 2011 and that the Liberals got 5% of the vote, this is a remarkable turnaround.

In Provencher, the Conservatives' Ted Falk led with 56% to 29% for Terry Hayward of the Liberals, another strong showing for the party (they had 7% here in 2011). The NDP followed with just 9%, while the Greens were at 6%.

And in Toronto Centre, which had the largest sample of the bunch by far, the Liberals' Chrystia Freeland led with 45% to 30% for the NDP's Linda McQuaig. That generally matches where the parties were in 2011. The Conservatives were third with 18%, while the Greens had 7%.

Problems?
The numbers look good for the Liberals and quite poor for both the Conservatives and New Democrats (losing Brandon-Souris would be a major blow to the Tories, not being in contention anywhere would be a strike against the NDP), but these polls are not without their potential problems.

First, there are the sampling issues that tend to appear in polls by Forum Research. Their IVR polls often over-sample older Canadians by a large degree, meaning that the samples of younger Canadians are smaller and thus potentially at risk of having larger errors. Those errors can then be amplified as the sample of younger voters is increased to its proper proportion.

Bourassa's sample was the least skewed, with 55% of it being over the age of 55, compared to the 40% of the voting population of the riding that is actually that old (at least according to the 2006 census, I do not believe the 2011 census has been broken down by riding just yet).

But the skew in the other ridings was significantly larger. In Provencher, 64% of the sample was 55 or older, instead of the 32% it should be. The sample in Brandon-Souris was 71% over the age of 55, when it needs to be 36%. And in Toronto Centre, the sample was 64% over the age of 55 instead of 25%.

Second, there is the issue of past voting behaviour. This is a tricky one, though, as the last election was over two years ago. Between then and now, voters have already cast a ballot in provincial elections and the population may have moved around (for example, all of the ridings report some respondents who voted Bloc in 2011, including those in Manitoba and Ontario). So, there is a natural amount of error that could occur.

Nevertheless, the samples do not match the voting results of 2011 very closely. For example, the sample in Toronto Centre reported a past vote of 51% for the Liberals, 24% for the Conservatives, and 18% for the New Democrats, instead of the actual result of 41% for the Liberals, 30% for the NDP, and 23% for the Tories. The samples are similarly Liberal in the other three ridings, while the NDP has been potentially under-estimated in Bourassa and Brandon-Souris. The Conservatives have been potentially under-estimated in the two Manitoba ridings.

If we weigh Forum's poll by how respondents said they voted in 2011, we get the following results:

This still gives the Liberals a lead in Bourassa and Toronto Centre, but Brandon-Souris swings over to the Conservatives. The gap widens in Provencher, while it shrinks in both Bourassa and Toronto Centre, with the latter being particularly close. However, it should be emphasized that weighing by past vote is not necessarily more accurate than Forum's overall numbers. We don't know, for instance, whether past vote is already part of their weighting scheme (my calculations may not be taking into account other weighting factors like age or gender).

What we can know
The Ontario by-elections in August demonstrated that by-election polling can be particularly hit-or-miss. The polls called the race rather well in Etobicoke-Lakeshore and Scarborough-Guildwood, for example, but missed by a significant degree in London West and Ottawa South. The polls in Windsor-Tecumseh, while picking the winner, were off in the details. That is not an especially strong record. If it persists, two of these four by-elections will be called incorrectly by the polls.

To take this into account, I've calculated some reasonable confidence intervals we can apply to the polls in these by-elections, based on how the polls performed in the Ontario and federal by-elections of 2012 and 2013, as well as the Westside-Kelowna by-election in British Columbia earlier this year.

The margin of error of polls always includes 95% confidence, implying that 19 times out of 20 the results are within the margin of error. In other words, there is a 20 to 1 shot that the polls will be right. But that is an assessment of how accurate the polls are of the general population. We cannot know for certain whether the polls are accurately depicting the voting intentions of the general population, since the entire population does not vote. In reality, recent by-election polls have been within their margin of error only about two-thirds of the time. That gives them a roughly 2 to 1 shot of being correct.

I will be averaging the by-election polls again as I did for the Ontario by-elections, but I will be applying 67% and 95% confidence intervals to the results. Put simply, those ranges reflect the degree of error that 67% and 95% of polling results had in recent by-elections. They should act as a good guide for what to expect on voting day, based on what the polls are saying.

Here the those ranges, applied to this series of Forum polls (the polls in Bourassa and Toronto Centre from the spring are too old to be included in the weighted averages, so the weighted averages are the same as the Forum polls across the board). You can click on these charts to magnify them.

The highlighted portions of the ranges note the party that has a winning result within those bands of support. As you can see for Bourassa, even the 95% confidence interval points entirely towards a victory by Dubourg if an election were held in the next few days.

In Brandon-Souris, both Maguire and Dinsdale could potentially win with the numbers in the Forum poll, at both confidence intervals.

Falk appears safe in Provencher, the by-election likely to hold the fewest surprises on Nov. 25.

Toronto Centre is more interesting, however. Freeland looks safe in the 67% confidence interval, but once that is increased to 95% her victory is in doubt. If the by-election were held this week, McQuiag would have an outside shot of winning - like the NDP did in London West this summer.

The starting positions have been established as the by-elections kick-off. The Liberals look secure enough in Bourassa, but we will need to see what happens to Laraque's support. The party also has the advantage in Toronto Centre, though the campaign is far from over. The Conservatives will need to work to hold on to Brandon-Souris, while Provencher looks safe. These four by-elections should be interesting.

I live in Brandon-Souris. If FORUMS numbers skew "older" then the fact Liberal Rolf Dinsdale is leading is beyond remarkable. Older, here, means staunchly Conservative. Added to that is Brandon, Manitoba's second largest city, is also in what can be considered prairie bible belt country. Guys like Rolf - a former punk rocker with decidedly progressive ideas and attitude don't fly too well.

I really hope the numbers are indicative of the vote as he would be a refreshing addition to parliament - versus the Con candidate, Maquire - an old, tire current MLA who actually lost this riding the last time he ran - when the Cons and Reform spilt the rightwing vote.

I'm going to be a a strategic voter. Rolf has my vote. And even though Harper has made it as difficult (weather wise) as he could - I'll be going out at the end of November on the prairies to cast my ABC vote.

That the sample skews older does not mean that the poll skews older - that 71% of the sample over the age of 55 would have likely been re-weighted so that it represents the 36% it should. The problem isn't that the poll skews older, it is that fewer people would have been polled under the age of 55 than there should have been, meaning there is more potential for error.

The 29% of the sample under the age of 55 had to be inflated to 64% of the weighted sample to get the proportions right, meaning any errors in the smaller sample of younger voters would be amplified when the sample is re-weighted.

Wikipedia has results for Brandon riding going back to 1953. In all that time, someone other than a PC or a Conservative has won once (in 1993), and the combined vote of the PC's and Reform/Alliance parties has been below 50% once (in 1988). The Conservatives losing there would be a huge upset. (Ironically, the Conservative MP from 1953 to 1980 was a Mr. Dinsdale.)

Age doesn't factor into my model, I was referring to the representativeness of Forum's sample. While it is almost certain that the proportion of voters over the age of 55 would have increased in each riding since 2006, the difference would not be enough to get the sample into line.

I wouldn't worry too much about where the George Laraque votes will end up going in Bourassa, A big proportion of people who were going to vote for him were doing so because of his star power among the Haitian voters of the riding. Not really because he was a Green. The Liberal brand there is strong due to Denis Coderre's lasting positive impression among visible minorities.

COMMENT MODERATION POLICY - Please be respectful when commenting. If choosing to remain anonymous, please sign your comment with some sort of pseudonym to avoid confusion. Please do not use any derogatory terms for fellow commenters, parties, or politicians. Inflammatory and overly partisan comments will not be posted. PLEASE KEEP DISCUSSION ON TOPIC.

Monthly Federal Polling Averages

Monthly Provincial Political Polling Trends (to Feb. 2015)

Details on the methodology of the poll aggregation and seat projections are available here and here. Methodology for the forecasting model used during election campaigns is available here.

Projections on this site are subject to the margins of error of the opinion polls included in the model, as well as the unpredictable nature of politics at the riding level. The degree of uncertainty in the projections is also reflected by the projections' high and low ranges, when noted.

ThreeHundredEight.com is a non-partisan site and is committed to reporting on polls responsibly.