Friday, May 30, 2014

Consensus or not in Ontario?

On the one hand, the new poll from Ipsos Reid for CTV/CP24 has moved into the ballpark of all the other surveys we have seen over the last week. It is looking like a close race, close enough that the gap between the two parties is within the margin of error - maybe even within a rounding error. On the other hand, the new poll diverges greatly with other surveys in showing a massive, crushing lead for the Progressive Conservatives among likely voters. The pollsters may be now gauging the opinions of Ontarians correctly, but unless things converge before the end of the campaign someone will be telling the wrong story when the votes are counted.

The projection has swung wildly again with the addition of this poll. But it is less a product of the tendency of the model to lurch back and forth with every single survey (Ipsos now occupies 57% of the projection, so plenty of room for the other numbers) than it is a sign of just how different the likely voter numbers are from the rest of the field.

The last EKOS survey had the Liberals at 36%. The last Abacus poll of likely voters also had the Liberals at 36%. And yesterday's Forum poll, which bakes in a turnout model into the overall numbers, also had the Liberals at 36%. So, Ipsos having the party at just 29% among likely voters causes a bit of a swing.

The PCs have now moved ahead again with 38.6%, a gain of 3.6 points since yesterday's update. Their range now stands at 37% to 42%. The Liberals dropped 3.8 points to 32.1%, or between 31% and 35%, while the NDP increased by 2.4 points to 23.4% (or between 22% and 25%). The Greens are down 2.2 points to 4.8%, or between 3% and 6%.

In the seat count, the Tories are up eight to 51, or between 39 and 57 seats, just potentially getting them over the majority threshold of 54 seats. The Liberals fell 13 seats to 36, or between 38 and 48, while the NDP picked up five to reach 20, or between 17 and 25.

So is this a case of the polls disagreeing with one another again? Not exactly. The polls are now in general agreement on the state of the race, in that it is close and somewhere around 35% for the two leading parties. But the way likely voters are estimated is different from one poll to the next, a difference that perhaps has a much more significant effect on the numbers than the methodological variations between telephone and online polling.

EKOS will be coming out with likely voter numbers before the end of the campaign, so we will see what method they are employing. Forum uses a likely voter model, but it is a 'secret sauce' and we know nothing about it. Abacus asks respondents a half-dozen questions relating to commitment and interest in the election, and bases their estimate on how likely a respondent is to vote on that. Ipsos asks survey takers how likely they are to vote, only counting those who say that 'nothing short of an emergency' would prevent them from voting to come to their likely voter tally. That number is tracking with turnout from 2011, so on the face of it could be an effective model.

This makes it not entirely about disagreement among polls but rather the models used for estimating turnout. The two are very different. We can only wait until June 12 to see which model was the best.

Ipsos was last in the field on May 20-21, and since then recorded a one-point gain for the PCs, who led with 36%. The Liberals were up three points to 34%, followed by the NDP at 23%, down five points. Support for other parties was up one point to 7%, and 17% of the entire sample was undecided.

Only the drop in support for the NDP would be outside the margin of error of a probabilistic sample of this size.

Among likely voters, the Tories were steady at 41%, with the Liberals and NDP each down one point to 29% and 25%, respectively. Stability here, then, which would suggest that while unlikely voters may be moving towards the Liberals (the party has gained in two consecutive Ipsos polls), those who are most likely to cast a ballot are sticking with their preferred choice.

Regionally, the PCs were in front in central, eastern, and southwestern Ontario, while the Liberals led in the GTA and the NDP in the north. The Liberal lead in the 905 region of the GTA is the first that Ipsos has recorded since the beginning of the campaign.

The disparity between the results among eligible and likely voters makes this poll a difficult one to put into context. It is certainly a relief to see the eligible numbers squaring with those from other surveys, as it means we now have a better understanding of what is going on. But the likely voter numbers are still extraordinary for the PCs, and like nothing other polls are showing (even those that explicitly or implicitly account for turnout). For now, we can simply say that turnout will be key, both in terms of pollster accuracy and the election outcome.

36 comments:

Turnout will be important, alright. But so will the last-minute impact of media reporting on polling. If the final polls show a significant Tory trend, watch for the opposite to develop, in response, on election day. Watch this space, everyone.

It seems a bit crazy that, with a 12% lead and sitting at 41%, the PCs wouldn't get a majority. I didn't look at the riding numbers, so maybe a lot of very close races that are predicted as lost, but still, it seems to me they should easily have their majority...

I was wondering about your riding numbers and Kitchener-Waterloo. I remember you mentioned the other day that you had Kitchener-Waterloo not very close and Eric had them going to the NDP. I was wondering what you were using as your baseline, are using the by-election results where the NDP won or if are using the general election results where the PCs won?

I don't use the regional numbers, so those can't influence my model. Region definition is sometimes too sketchy and number of people polled is too small.

As for by-elections, I don't use those either. They are different beasts entirely with messages being sent to parties where, very often, things go back to normal during a general election. I could potentially add them, but count them less than general elections, but that's only my second election (Québec being the first), I'll see about upgrading it after.

Seems to me that Ipsos' methodology is biased towards the conservatives. By taking only those most likely to vote, are ignoring those only somewhat likely to vote, they are capturing only a part of the population that will actually vote, a part which we know leans more towards the conservatives than the rest of the actual voting population.

Seems that it would be better to model for each respondent a probability of voting and use those as weights. That way you're not ignoring big swaths of the population.

I think the counter argument is that a lot of people in opinion polls seem to over estimate their likelihood of voting. Turnout is likely to be below 50% for this election so a model that only counts the 51% of respondents who said they are certain to vote seems like a pretty reasonable method to try and identify likely voters. Having said all that, I suspect someone who is willing to do an online poll about an election is probably a bit more interested in politics and therefore more likely to vote, so I could see a rational for increasing the likely voter model a little bit. However, the problem is this increases your projection of how many people in the panel will turn out from 51% to 81%. (the next question reads: I would do my best to vote, but sometimes things get in the way). This just seems to high for me. I suspect you may be curious, as I was, what happens If you add these individuals into the likely voter model. The results, if you add these next individuals into the likely voter model and count them fully, is PC 38%, Liberal 33% NDP 23%. If you want to weight them the result would be somewhere between those numbers and the numbers Ipsos provides depending on how heavily you weight them.

Thanks for your efforts, Eric. Again, the polling firms seem to be operating within their own biases. What gives? Left leaning Ekos has the Libs in front and right leaning Ipsos puts the Tories in front. The political affiliation of the firms is now a greater independent variable than recency/time. Maybe that should be factored into your weighting system! :-)

I would love to know how Ipsos is finding their respondents. Internet is not like IVR, you can't just go into 1000 random homes and ask for their opinions over the internet. Obviously it's a sampling of people who are hitting particular websites. And from the last Ipsos poll, we can only conclude it's the same websites. Thus you're only going to get the people who hit those websites. Not exactly random.

Ping pong. Welcome to Ontario where we're watching a riveting game of three-way table tennis. Who's ahead? You decide! Today it's, oh, never mind, it just changed again. I'm pretty sure the average voter out there is desperately looking for Premier Above, first name "None of," last seen in the mid-80s in Ontario ....

I thought the mess with the PC's being unable to read at a grade 3 level (mixing up years with jobs) would kill them, but it seems to have had a media life measured in hours vs the latest story on the Liberals (leaked by the PC's and saved for just such an emergency it seems) blowing over $300 million to help out MaRS by buying an office tower.

Sigh. The way the media lets the big parties (especially the PC's in this case) manipulate them is scary. Btw, I do feel both are scandals worthy of coverage but it blows my mind that a party could be taken serious that makes such a basic mistake as the PC's did on their core platform.

Probably has something to do with the Liberals already being scandal ridden, people still remembering the billions wasted on the gas plant cancellation, etc etc. The Tories aren't in power with no track record to go on, so any debate over the validity of their platform is just an intellectual exercise based on opinion and conjecture among people of differing political viewpoints. Contrast this with the MaRS "scandal" which is yet another situation where the Liberals are perceived to be wasting taxpayer money with little regard for the public. The media loves a good scandal.

There's no scandal in a platform being torn to pieces, that's par for the course during every election. There is a scandal when a government is perceived to have 'wasted' more money. And the media loves a good scandal, that much should be obvious.

How to do you weight the polls given that some have all voters and some have likely voters? To me it seems they are measuring different things so should have a different impact on the model. The everyone surveys should be a more accurate general reflection of overall public opinion because less is done to the numbers and therefore less added statistical error. On the other hand they are not usefully measuring how people will vote.

Is there a track record weighting that comes into play that is different between the Likely and Eligible voter models? There is a much longer track record for the Eligible. Do you have a default reduction weighting for new polling companies or for new methodologies?

I work in the OPS, and I'm junior enough that a big jobs cut may see me unemployed, so naturally I'm drawn to any site that projects the election. I have to say, while I love the site, dude, you're killing me. One day the Liberals are up, and I think, hooray, I'm safe! The next, I wonder how long EI benefits will last.

I wish you the best of luck whatever happens but, being a junior civil servant may actually save your job. When I worked for the feds during a time of layoffs it was the senior people who the Government wanted out because, they get paid far more. 1 senior layoff usually is enough to employ two junior people. Plus due top the way the pension system used to work federally if you work 35 years you receive 75% of your best 5 years as pension but, if you work 34 years you only received 60% plus a small top up. Nowadays everything is based on years; 1 year equals 2% of salary so if you work 25 years you receive 50% of your salary as a pension.

My (I think informed) intuition is that Ipsos' likely voter screen is terrible. The problem is that while the "emergency only" people will mostly vote, so will many of their other respondents (their unlikely voters), especially in the second category ("do my best"). The fact that the first category matches expected turnout well is because the people in their panel will certainly vote at a much higher rate than the general public because people who do online political polls are considerably more interested in politics than average. One hint at the problem is that Ipsos has 58% of men but only 44% of women as likely voters, when women consistently vote at least as much as men. I think Ipsos may be headed for a rude surprise on election day.

I don't see how NDP numbers increase when using likely voters. The evidence from polls and BC suggest the opposite. We know the NDP does better among the young and we also know from Elections Canada and others that only about 15% of the 18-24 year olds vote, we know that 25-35 year old have the second lowest turnout rate. It makes me think that these "likely numbers" are likely not correct!

It has always puzzled me how online can be accurate at all isn't there an inherent self-selection bias? What's Ipsos' track record with this method? I'm not discounting their results at all but they have since the beginning had much different results than the rest of the pack. Either we've got a revolutionary new model for polling in Canada, or they're dead wrong. It'll be really interesting to see which it is.

The problem with all polling now is that it really, really hard to get a true random statistical sample.

For online polling what matters is how large pool they have to draw from and how they populate their pool. EKOS and Ipsos work to build a large pool of people and use methods of recruitment that offer diversity. Is it perfect? Likely not, but it is what works at the moment.

With phone polling the companies have a much easier time get people 55+ to respond than any other demographic which means they need to weight their results a lot to reflect the over all public.

Can someone else verify that the Ipsos likely voters N is really 439, not the 868 which is the full sample (and which had only a 2% gap) and which is the number in Eric's table. And, perhaps Eric might explain why only the "likely" numbers are used in your running calculations and not the full sample? it was the same poll, done at the same time, wasn't it?

As I've explained many times, I use the likely voter numbers from any pollster when available, because I am trying to project the outcome of the election in which only a portion of the electorate will vote, rather than the opinions of 100% of the electorate.

For the poll table, I record the total sample size of the poll, including undecideds.

Sorry, but it seems to me from Ipsos table that the N for the likely also includes undecideds. It would be an easy thing to simply put the N for the subtotal used for the likely next to the total N (in parenthesis) so as not to unintentionally mislead since the N for the likely is given and is clearly NOT the N for the whole survey.

Also, can someone explain why Ipsos likely %s vary so much from the others? Is their likely sample demographically different in some significant way?

Eric, have you added the forum northern poll into the model? I don't think it will make a huge difference as you have the same results in your riding predictions as forum but I am curious if you've included it.

Urban Ottawa and Kingston haven't voted PC in a million years (Ottawa-Vanier has never voted anything but Liberal since its creation almost 100 years ago). If they over-polled from Lanark county and the Ottawa Valley, then this puts the East numbers out of whack in this poll.

In addition other than Kingston Eastern Ontario voted solidly conservative. Urban Ottawa either has 4 or 7 ridings depending on one's definition. Ottawa South is a swing riding as is evident from the 33% the Conservatives took in the 2011 election. Provincially Ottawa South was solidly Conservative until Dalton Sr. usurped the riding in1987.

The reality is Tories do better in urban areas than many on the left wish to believe. 56% in the East may be an over-sample or an outlier but, it is probable the PCs will garner 30% of the vote or better in "urban Ottawa"

COMMENT MODERATION POLICY - Please be respectful when commenting. If choosing to remain anonymous, please sign your comment with some sort of pseudonym to avoid confusion. Please do not use any derogatory terms for fellow commenters, parties, or politicians. Inflammatory and overly partisan comments will not be posted. PLEASE KEEP DISCUSSION ON TOPIC.

Details on the methodology of the poll aggregation and seat projections are available here and here. Methodology for the forecasting model used during election campaigns is available here.

Projections on this site are subject to the margins of error of the opinion polls included in the model, as well as the unpredictable nature of politics at the riding level. The degree of uncertainty in the projections is also reflected by the projections' high and low ranges, when noted.

ThreeHundredEight.com is a non-partisan site and is committed to reporting on polls responsibly.