Captain Ed is a father and grandfather living in the Twin Cities area of Minnesota, a native Californian who moved to the North Star State because of the weather. He lives with his wife Marcia, also known as the First Mate, their two dogs, and frequently watch their granddaughter Kayla, whom Captain Ed calls The Little Admiral... [read more]

Note: This originally was going to be an update on my post from yesterday noting the desperation at the Kerry campaign and their plan to bring in Bill Clinton to boost Kerry on the stump. After being challenged in the comments about polling numbers, I intended on giving a brief explanation ... but you know how 'brief' I can get ...

The recent CBS News poll, published May 24th, showed a dramatic increase in support for John Kerry and an equally significant drop in George Bush's approval ratings. It looked as though John Kerry had finally achieved some traction in the race, using his new advertising campaign to attack Bush at a vulnerable point and building some momentum towards the convention. CBS' poll got massive exposure in the mainstream media and generated a huge amount of buzz in the blogosphere. The poll only has one problem.

It lies.

Just to put the polling in context, take a look at the Rasmussen poll, which shows a continual dead heat between Bush and Kerry, even on a head-to-head basis, using a sample of 3500 likely voters, with the last data reported 5/29. Quinnipiac has Bush ahead by one point in a three-way race, 43-42-6, as of 5/26, using a sample of 1160 registered voters. While you're there, take a look at the trends; Bush's favorability rating has stabilized at 40% over the past two months, while Kerry's has dropped four points and is below Bush's number.

The CBS poll, on the other hand, uses only 1113 registered voters, broken down in an unusual manner: 346 Republicans, 390 Democrats, and 377 independents. Since when are there that many more Democrats than Republicans? The poll then shows its "weighting" (although it doesn't explain what it means), and the numbers get even worse: 330-R, 401-D, 381-I. According to the University of Pennsylvania in 2003, Republicans accounted for 32.5% of the registered electorate, while Democrats accounted for 33.7%. In a sample of 1113 voters, you would then expect to see 361-R, 375-D, 376-I. The result of CBS's sample is to throw off representation for Republicans by 8.6%, while bolstering Democrats by 7% and independents by 1.3%, using CBS' weighting.

The outcome, therefore, should hardly surprise anyone -- overreported Democrats are surely likely to support John Kerry, while the underreported Republicans depres Bush's ratings. It could hardly escape anyone's notice that CBS' result differed dramatically with other polling taken at the same time, using larger samples and better vetting, such as Rasmussen's 3,500 likely voters -- or could it?

Only RealClearPolitics remarked on the CBS sampling problems, noting that it represented a solid trend at the Tiffany Network:

If you go back, as I did this morning, and look at job approval numbers from the same group of pollsters for the first five and a half months of 2004, you can see the consistency of the CBS/NYT bias more clearly.

In every instance except one this year (and a very iffy one at that), CBS/NYT produced the worst job approval number of any of the three polls during a comparative time period.

And then Tom Bevan gives you a handy table demonstrating the phenomenon. All other news sources, however, gave CBS numbers straight, without any mention of its rather anemic sample -- you'd think a broadcast network could afford to poll larger -- its unreliable sample type, and its poor representation of the electorate. The lesson is that in order to analyze the polling numbers, you have to actually do some work and check the samples.

Trackback Pings

» Beware Polls from David M
A CBS News poll published May 24 states, "Overall, 49 percent of registered voters now say they would vote for Kerry, 41 percent for Bush."
The poll, which contained other bad news for Bush as well, saw a lot of media play.
But Captain's Quarte... [Read More]