Friday, September 17, 2004

Exclusive: The hidden demographic

Polls are important, but limited. To evaluate the usefulness of polls you most decide what polls are measuring, their limitations, and if they are getting the whole picture.

I believe that the polls are missing a potentially large demographic. Everyone who has graduated from college since 2001. Their were, "1,244,000 bachelor's degrees conferred in 2000-01". That is about four million people who were in college when the twin towers were struck. I think this demographic is very important because of the prominence of 9/11 in this demographic's mind and the several problems in measuring the sentiment of this group.

The problem with polling this group are:

They may not show up on Most likely voters polls. College students are not know for their voter turn out, and this group has not had a chance to vote in a national election since 9/11.

They address may not be included on polls as much as other voters who have a long standing address and a listed phone number. Their contact info might be present to a lesser extent. Most probably do not own homes and probably live with parents or in apartments. Some may not have home phones. I have a cell phone, and do not have a home phone.

Once their contact information was included they might stayed below the radar of the polling agencies. How many 22-26 year olds are home when the pollers call. I would guess to a lesser extent then the 30 year old with kids.

I think the polls are probably trying to poll this demographic comparable to percentage that voted in the 2000 election. If 10 percent more of this demographic turns out, that is about 400,000 voters. If this demographic turns out in force, seventy percent, that is about 800,000 more votes being cast. Enough to decide the election, maybe.

3 comments:

I used to work in a technical capacity at a political polling phone bank and I can tell you that you have correctly evaluated a number of problems with political polling and that these problems are getting worse.

The science/art of political polling of course is to extimate from your sample how accurately the sample reflects the population as a whole with the population in this case being those who will actually turn out to vote.

Cell phones as primary phones are one particular growing phenomenon that is troubling to polsters as this is a signifigant demographic, with similar traits, that is difficult to evaluate.

On the other hand, contact info or unlisted phone numbers is less of a problem. The majority of political polls use random digit dialing to generate their sample. This means that they randomly generate the numbers that they call so even unlisted people get phone calls.

The methods used to evaluate the final results of a poll and adjust it to match known demographics are a large factor in what causes different polls to have very different numbers.

Cell phones don't get called because only the last 4 numbers are randomly generated. The 3 digit exchange numbers (and area codes) are chosen specifically to use the demographic data on those echanges.

As a pollster, I would want to make sure I hit all areas that I was covering in my poll proportionally to the number of people in each area. This will give me a more representative sample. Cell phones, since they are not tied to geography are useless for this so cell phone exchanges are never used in random samples (well, never as far as I know).

Caller Id and Call screening is also a problem for pollsters, lowering the sample accuracy and possibly biasing the results. For example, if wealthy people are more likely to screen their calls than poor people you end up with a survery biased toward the poor. If the pollster believes this is going on, he will adjust his poll results to reflect this bias.