Wednesday, November 28, 2007

Waterboarding the numbers

Yet another marginally interesting survey that says something -- just not what the heds claim when they say "survey says." Here's how two papers treated it:

SURVEYBlack voters favor ClintonBut if Obama wins in mostly white Iowa, that might change mindsWASHINGTON --Black voters may be leaning toward supporting Hillary Clinton for the Democratic nomination even though they prefer Barack Obama, because they're dubious that America is ready to elect a black president, a new survey suggests.(Charlotte Observer)Neither the hed nor the lede is strictly true. The survey doesn't suggest that black voters are leaning one way despite their underlying preference. That suggestion comes from an analyst at the outfit that conducted the study:"I think there are a lot of black voters who think Hillary Clinton has a better chance of being elected president," said David Bositis, senior policy analyst for the center, which specializes in analyzing issues important to African-Americans. "They're basing this thought, this feeling, on their own experiences. African American voters think . . . 'there's no way in the world a black candidate is going to be elected president.'"He's welcome to his opinion, and he might even have a point. What he doesn't have (or, at least, doesn't present here) is anything in the way of empirical evidence to support his opinion. So the deck needs to be attributed and the lede needs to be tied to the analyst's judgment, rather than the data. Which are also the main problems with the display treatment at the Free Press:

Iowa win vital for Obama, poll saysIt could be proof for black voters

You can make a fair case that an Iowa win is equally "vital" for everyone (though if the news media that piously sign the Stop Me Before I Horserace Again pledge every year would just shut up about it, the silly thing might well recede in importance). It's possible, at a stretch, to assemble conclusion like that from a poll -- just not from this poll. And, of course, any "could" hed, like a "might," can be phrased as a "might not" with little or no loss in meaning.

Charlotte's main hed -- "Black voters favor Clinton" -- has a similar problem, and it also stems from ignoring the most important question you can ask about a poll: What did it measure?

The poll didn't ask a straight-up "whom would you vote for" question. Rather, those surveyed gave Clinton an 83 percent favorable rating and Obama a 74 percent favorable rating.

So we don't know who(m) black voters "favor"; all we know is how they respond when asked to rate candidates favorably or unfavorably. It's not a bad question, but it isn't the one claimed in the hed. Nor does it address proof, nor does it address the relevance of the Iowa festivities. The numbers aren't lying. They're just being gently told what might happen to their properties and families.

It's E-Z and fun to make fun of Fox's polling, because Fox and Opinion Dynamics jump through such tormented hoops to get to where the paymaster wants. But at the end of the day, fibbing with data is fibbing with data. Don't confuse "advocate says" with "numbers say." That's cheating, no matter whose side you're on.