Thursday, October 23, 2008

Longer Field Periods and Round Numbers

If there is one thing that I have found to be an absolute truism as a pollster in this year's election, it's that Barack Obama's supporters are harder to get in touch with than John McCain's and Hillary Clinton's.

During the early stages of the primary we released a lot of one night, one shot polls and consistently underestimated Obama by a whole heck of a lot (even if we still did better than a lot of our colleagues in places like South Carolina.)

So we started doing longer field periods and more callbacks starting with Wisconsin, and our polls were pretty much on the mark for the rest of the primary season (with of course one exception.)

This trend has continued in the general election. We conducted half the interviews for our Florida poll this week last Thursday night. The first night McCain led by 2. The data based on callbacks over the next three days gave Obama a 4 point lead.

It was a similar story in Colorado two weeks ago. The first night Obama led by 5. The rest of the field period he led by 15.

I don't think there was movement in Obama's direction in subsequent days either of those times. It's simply a matter that you have to try more times to get his supporters on the phone.

Which leads me to today's Quinnipiac Ohio poll showing Obama up by 14. That's a big difference from Rasmussen and Mason Dixon's polls earlier in the week that showed McCain winning, and I think part of the discrepancy can be traced back to the length of field time. Rasmussen's was a one night poll. Mason Dixon's was a two day poll. They couldn't have done that many, or any, callbacks to get folks on the line who weren't at home on the first shot. Quinnipiac was in the field for six days, and I'm guessing they managed to get a lot more of those elusive Obama supporters polled when they tried for the third or fourth time.

Another thing to consider when looking at polls is the number of respondents- not just whether it's a large or small number, but whether it's a nice even number or not. If it's a nice even number a lot of the time that means a pollster is just getting a set number of interviews and then stopping. That may have the potential to undercount support for Obama, again because his folks are harder to find. If it's a random number, that's an indication that folks have their sample and they've decided they're going to call everyone in it 'x' number of times and try to get as many respondents as they can get.

We're not going to be putting out Quinnipiac sort of numbers on Ohio this weekend: more like Suffolk numbers. But from my experience anyway the issue of field periods and round numbers may help to explain at least some of the 16 point difference you see between Rasmussen and Quinnipiac.

14 comments:

Thanks for the perspective, Tom. I didn't even think about the fact that pollsters would keep trying to contact the same house rather than just call a different number. So you have a list of 700 people that you absolutely know ahead of time you are going to contact, and keep trying until you contact all of them?

Thank you so much for this posting. After looking through both the Q Findings and the Battleground polling, especially Indiana, I'd developed a hypothesis that McCain voters are simply not responding anymore to pollsters somehow, otherwise how do we explain such rapid motion across the board. I had though it may be that they're despondent and simply hanging up, yet they'll still vote. My GOP friends won't even talk about the election anymore, they're so depressed about it, but I would be money they will vote no matter what. This response bias makes me worried in looking at polling results like Indiana +10. It makes you wonder if that sample isn't somehow tainted, and if Indiana is wrong, then that means all of those states are off too, and PA, WI are both closer, as is IA than we may know.

Thanks for your explanation, it totally counters this hypothesis. Though one still has to wonder how Indiana came to poll +10O, bad weighting?

I looked at the Big 10 polls in question. Weighting didn't seem to be an issue - what was the issue was that their methodology was poor in that they asked all sorts of questions before the presidential race - opinion on Bush, what direction the country is going in, what their most important issue was, state of the economy, etc. A lot of these questions would really skew the results - favored methodology should be to ask the presidential question immediately, as they would in the actual election.

Of course, results like this do suggest that it would be a very effective tactic to have volunteers standing outside polling places on Election day holding up signs asking, "Do you think the country is going in the right track or the wrong track?"

I'm wondering what the probability is that some of those being called were either in a line waiting to early vote (and impossible to reach if they didn't have a cell phone or if the poll only called their land-line #) or involved in a GOTV effort, and thus away from their cells or land-lines.

This effect would be greatest, of course, in an early voting state with long lines.

I would suspect this might knock off a couple % off the Obama figures...and even worse because these are "banked votes".

Tibet Travel Dot Info offers first-hand and up-to-date tibet travel information - tours, maps, climate, transportation, accommodation, travel tips,We are offering tibet tour in great discount,Tibet tours by local Tibet travel agency - Access Tibet Tour help you travel to tibet with ease, with tibet tour guide, Tibet Adventure Travel packages!U-Way is a new travel company Specilized in china tibet tours, furnish cheap tibet tours in 2009, if you wanna travel to tibet, please believe in u-waytours, let we work-team service for you.. We are provide tibet trekking routes,Tibet travel permit and overall service of travelling to TibetAs we know,wow gold guide and on our website.Free gold guide We have available stock of gold on most of the servers

Do you know seal cegel? I like it. My brother often go to the internet bar to buy sealonline cegel and play it. After school, He likes playing games using these seal online cegel with his friends. I do not like to play it. Because I think that it not only costs much money but also spend much time. One day, he give me many cheap seal cegeland play the game with me. I came to the bar following him and found buy seal online cegelwas so cheap. After that, I also go to play game with him.