Friday, August 23, 2013

Why forecasting Quebec's Liberal seats in 2015 won't be easy

Sure enough, after La Presse put out the numbers for CROP's latest provincial poll on Wednesday, the inevitable federal Quebec numbers followed yesterday. They showed the Liberals continuing to be well ahead in the province, with the New Democrats taking a step backwards to a new low in CROP polling. The Conservatives were up significantly, but are still below where they stood on election night in 2011.

To briefly cover the shifts since CROP's last poll from mid-June, the NDP was down five points and the Conservatives were up six, while the Liberals were down an insignificant single point and the Bloc Québécois was up two.

Regionally, the shifts worth noting were a spike for the Liberals in the Montreal suburbs, NDP drops among non-francophones and on the island of Montreal, and Conservative gains among francophones, in Montreal, and in Quebec City. On who would make the best Prime Minister, Justin Trudeau was up two points to 31%, Thomas Mulcair was down eight points to 23%, and Stephen Harper was up two points to 12%.

What I'd like to discuss in detail, however, is the challenge that 2015 will pose in forecasting the seats the Liberals can win in Quebec.

Since Trudeau boosted the Liberals into first place in the province, this site has only rarely given them the number of seats one would expect with a double-digit lead. I have explained this due to the lack of a Liberal base outside of Montreal which limits the projection model's ability to move seats over to the Liberals. When the party was under 10% in a riding - which was very common - tripling their vote still only increases their support in that riding to 30% or less. That is not enough to win.

When a party goes from a very low level of support to a very high one, projection models can give wonky results. But not necessarily - despite the New Democrats surging from 12% to 43% in Quebec between the 2008 and 2011 election, the projection model was able to handle that without issue. With the right province-wide numbers plugged into it from 2011, the model would have given the NDP 60 seats (they actually won 59).

Why was the model able to do this? Because the NDP went from uniformly low support in every region of the province to uniformly high support. By my rough estimate, the NDP had between 11% and 14% support in the four regions defined by CROP in the poll above in 2008.

That uniformity continued into 2011. The New Democrats took about 38% of the vote on the island of Montreal, 47% in the surrounding suburbs, 40% in Quebec City, and 44% in the rest of the province. Across the board, support for the party tripled or quadrupled. With that sort of uniformity, the projection model had no trouble with it, particularly since support for the other parties decreased so much.

It is a completely different story for the Liberals. They had virtually no support in Quebec City in 2011, with only 7% of the vote. They did not fare much better in the regions of Quebec with 10%, but took 14% in the suburbs of Montreal and 27% of the vote on the island of Montreal itself.

What that means is with CROP's latest poll, we have the Liberals quadrupling their vote in Quebec City and the regions of Quebec and tripling it in the suburbs, but less than doubling it on the island of Montreal. That causes trouble for a swing model. With an increase from 14% to 41% province-wide, the model triples the Liberal vote everywhere. In other words, it is increasing Liberal support by too much on the island of Montreal (wasting a lot of votes for the Liberals) and not enough outside of the city and its suburbs (not giving the party enough votes).

To address this problem, I worked on the projection model to make it capable of taking into account the regional data that polling firms like CROP and Léger Marketing routinely release for the province of Quebec. Using this regional data makes a big difference.

(Seat projection for Quebec)

Using only CROP's province-wide numbers, the model would give the Liberals 36 seats to 33 for the New Democrats, eight for the Conservatives, and one for the Bloc Québécois. When using their regional data, however, the Liberals are boosted to 46 seats while the New Democrats fall to 23. It means the model is able to take into account the Liberals' disproportionate growth in support outside of the Montreal area.

The variations between the two sets of projections occur throughout Quebec. For the Liberals, using province-wide data would under-estimate their seat count in the regions by 10, while over-estimating the seats the NDP could win in Quebec City (by four) and the regions (by seven).

Using the regional data, the Liberals would win 14 of 18 seats in Montreal and nine of 17 in the surrounding suburbs, while also winning 21 in the regions of Quebec. They would even win two in Quebec City (in close three-way races).

The New Democrats take 11 seats in the regions, eight in the Montreal suburbs, and only four on the island, while the Conservatives win six seats in Quebec City and two in the regions. The Bloc retains only one seat.

The conventional wisdom that the Liberal vote will be inefficient in Quebec may not be as wise as it seemed. CROP has consistently shown that the Liberals have more than a little life among francophone voters outside of Montreal, and that they do have the ability to win seats outside of their traditional bases. And if the Liberals look like a winning party in the province, they will have an easier time attracting quality candidates, making it in turn easier to win.

But the disproportionate shift in support in Quebec will make forecasting the results in the province in 2015 quite difficult. On the one hand, most polling firms do not release data this detailed for the province, meaning that swings based only on the province-wide results should under-estimate the number of seats the Liberals will win and over-estimate the number that could go to the NDP. On the other hand, projections calculated with the regional numbers from CROP and Léger Marketing, while having the potential to be more accurate, will be based on small sample sizes and so will be more prone to errors. And unlike the national numbers, where there are similarly small samples for the Prairies and Atlantic Canada, there will not be as many polls to smooth out the noisy data. CROP, for instance, did not report in the final 12 days of the campaign in 2011 and Léger only put out national numbers.

This will be something to keep an eye on. The Liberals have retreated to the cities in the last few elections, but in Quebec they are gaining support in the countryside. Regional data for other provinces, particularly Ontario and British Columbia, would be especially useful to determine whether the gains the Liberals have made are similarly disproportionately distributed. There are a number of reasons why the 2015 election is looking like a difficult one to forecast. This might be the most important one.

18 comments:

I think we have seriously underestimated the Greens in Bourassa. I expect them to finish in the top 3, possibly even 2nd. He is considerably more than just a former star Canadiens or Haitian. More importantly opponents will find it difficult to attack him without offending a constituency who view him as an immigrant inspiration story. Once a public figure emerges and draws attention to Green policies, people actually take notice how good they are.

It's possible that the Greens can improve their score in Bourassa with GL running, but it won't be easy. The Greens finished at the bottom in the riding last time and there isn't much of a base there. May was able to finish 2nd in Central Nova & London North Centre, but it wasn't until her 3rd try in the environmental heartland in Saanich-Gulf Islands that she was finally able to break through. They did that by a huge concentration of resources in one area and a large Green volunteer force flown in from all over the country to be May's ground forces. Will be interesting to see whether the Greens take more away from the Liberals, NDP etc. in Bourassa. Strong Green campaigns in the past (those mentioned above) have tended to take from multiple parties.

It amuses (and amazes) me to no end to see how mes co-citoyens de La capitale nationale so readily drink from the same right-wing glass both at the federal and provincial level. I seriously doubt that François Legault would appreciate his party being considered in the same breath as the Conservative Party of Canada.

As PQ finance minster, Legault happily presided over the same corrupt mess as all the others. He's really not in the position to talk about cleaning up anything in Quebec.

The right-wing nature of Quebec City puzzles. It is the greatest benefactor from a bloated Quebec bureaucracy that systematically transfers wealth from Montreal to Quebec City. They're threatening their own redundant jobs if they want cuts in government spending.

Well this seems to contradict my recent hypothesis that BQ support is closely tied to PQ support: the CROP poll confirms my suspicion that PQ support is up but pegs BQ support quite a bit lower than recent nation-wide polls.

Éric, perhaps you're aware that the recent Nanos poll commissioned by Bell and Telus contains some national vote intention results; see p. 4-6 of the tabulations below:

http://www.nanosresearch.com/library/polls/POLNAT-W12-T582E.pdf

I'm not sure if you'll opt to disregard these numbers since the poll was commissioned by "private interests", but what really blows me away is the abysmally low support for the BQ: 12.2% provincially / 2.5% nationally. I think that must be just about the worst result they've ever gotten!

Ha! That's interesting. I haven't heard anyone discuss Labeaume's future intentions so I don't have much of a clue which way he would lean. It seems he likes being the mayor too much to give that job up.

Well, interestingly enough Justin Trudeau & Régis Labeaume just met a day ago during the Quebec City tour JT is doing. Not sure if Labeaume is considering a jump to federal politics, but he did seem to turn on the Conservatives in 2011. He initially seemed supportive but then spoke against them when he felt he betrayed Quebec over the hockey arena issue. Now the NDP represents the area, but perhaps he will change his loyalties back to the CPC, or be open to the Liberals if they do well. I think like many Mayors, he goes with whatever party he thinks will do best for his region.

Expect for Daniel Paillé to be totally alone in 2015. He is a big unknown, lost his seat in 2011 in Hochelaga (a souvernist stronghold). BQ's Mouriani will lose her seat in Hunstic.

Justin has openly positioned himself to be the defender of minorities and has openly objected Pauline Marois' ''Charter of Values'' while Thomas Mulcair stays mum not wanting to lose his nationalistic coalition.

This vocal Justin will gain minorities who voted NDP in 2011. Mulcair is going to chose to keep nationalists instead.

It's true that the Liberals don't have much of a base outside Montreal right now, but if they significantly improve among Francophones under Trudeau compared to where they were under Dion & Ignatieff, doesn't that mean they can win seats in other regions, like Jean Chretien did?

Anyway, I agree it will be difficult to predict Quebec seats far in advance, because it could depend on the vote splits between all the parties, as it does in the GTA and other areas. Many ridings in Ontario were determined by only about 20-25 votes in the last election!

What I'm most interested in is the short-term impact of where the parties in Quebec area. No one can know who will be on top in 2015, but if the Liberals have such a huge lead with non-Francophones right now, doesn't that give them a big edge in the Bourassa by-election? That will be the first 'election' in Quebec so to speak.

I think you ought to reconsider your proportional-swing projection model. I suspect a uniform-swing model, while not perfect, is probably better, and with some sophisticated number-crunching I could figure out a model better than either.

The model you use supposes that a gain from 2% to 4% is as much of a breakthrough as a gain from 30% to 60%. I don't think that's anywhere near as impressive. The uniform model considers that a swing from 30% to 50% -- or a collapse from 50% to 30% is as remarkable a change as a swing from 10% to 30% or from 30% to 10%. That's probably also not true, but may be less inaccurate than the supposition behind your model.

Here's the research I envision -- has anybody done it before? Aggregate data from all parties, all ridings, for several past elections. Use it to figure out, empirically: if a party had p% in one election, what is the standard deviation s in their swing in the next election? Once you figure that out, project the swing this way: whatever multiple of s, applied to every riding, gives the party overall the same percentage as the poll data you are projecting from. So, if the polls suggest that the Birthday Part has 29% of the vote in a region, and adding 0.6s in every riding -- keeping in mind that s is not a constant across ridings -- gives them 29%, then add 0.6s to their totals.

I'd be really interested to know what kind of function would calculate (or estimate) s in terms of p. Of the two models you compare at http://www.threehundredeight.com/2011/03/methodology-of-projection-model-and.html a constant s would give the uniform model, while s directly proportional to p would give the proportional model. My guess is that s increases with p, but less than linearly, and probably hits a maximum when p is around 40-50%

Possibly, I don't know. But I've used this sort of model for nine elections now, and the problem has always been the polls, not the seat projection. There is more accuracy to gain from figuring out what to do with the polls, I think.

COMMENT MODERATION POLICY - Please be respectful when commenting. If choosing to remain anonymous, please sign your comment with some sort of pseudonym to avoid confusion. Please do not use any derogatory terms for fellow commenters, parties, or politicians. Inflammatory and overly partisan comments will not be posted. PLEASE KEEP DISCUSSION ON TOPIC.

Details on the methodology of the poll aggregation and seat projections are available here and here. Methodology for the forecasting model used during election campaigns is available here.

Projections on this site are subject to the margins of error of the opinion polls included in the model, as well as the unpredictable nature of politics at the riding level. The degree of uncertainty in the projections is also reflected by the projections' high and low ranges, when noted.

ThreeHundredEight.com is a non-partisan site and is committed to reporting on polls responsibly.