Features » September 14, 2007

Trending Toward Inanity

If you wanted to ruin the political career of Mark Penn, Hillary Clinton’s chief pollster and strategist, here would be one way to do it: First, create some sort of artifact bearing his name that you could use to tank his reputation. A book would do perfectly. Title it something buzz-wordy and superficial, like Microtrends, though perhaps that’s too heavy-handed. Fill it with vapid koans, like “small is the new big” and “the biggest movements in America today are small.” To make it seem authentic, you’d want to ape Penn’s long-standing affection for combining demographic salami-slicing with cutesy-naming (this is the man who foisted “Soccer Moms” upon our weary lexicon), making each short chapter an exposition of ever-more absurd groups–think “Archery Moms,” “Old New Dads,” and “Ardent Amazons.” Finally, assert their importance through wild and empirically unsupported speculations. That last would be the key: You’d want the methodology so wild and slipshod, so transparently flawed, that no one would trust the analyst ever again.

Astonishingly, Penn himself has done exactly this. His new book Microtrends is so bad that the question–in a fair world–isn’t whether it will destroy his own reputation, but whether it is so epically awful as to take the entire polling industry down with it.

First, a bit of full disclosure: Unlike everybody else in Washington, I have never met Mark Penn. This, I am assured, is to my enduring discredit, as he’s apparently a lovely individual, and if I only knew him, I would understand that his protection of a union-busting division within Burson-Marsteller, the PR firm of which he is the CEO, isn’t evidence of anti-union feelings at all. Nor is Penn’s ceaseless advocacy for a cautious, hawkish, pro-corporate, don’t-rock-the-boat Democratic Party a function of his beliefs, corporate background or clients. Instead, it’s merely “The Numbers.” Indeed, nothing Penn says or does can be questioned, because he’s just there to give us The Numbers. His personal thoughts are immaterial.

At the same time, his personal thoughts matter. As the Washington Post recently reported: “In the four months since Clinton officially became a candidate, Penn has consolidated his power, according to advisers close to the campaign, taking increasing control of the operation. Armed with voluminous data that he collects through his private polling firm, Penn has become involved in virtually every move Clinton makes, with the result that the campaign reflects the chief strategist as much as the candidate.” Even there, though, the Penn mystique persists: Penn’s power, we’re told, comes from his “voluminous data,” not his opinions. To argue with Penn is to argue with The Numbers. And you’re not against Numbers, are you?

That’s the Penn defense, and he and his friends have long stuck to it. “Mark is somebody who is very, very comfortable with quantification,” enthused Doug Schoen, his polling partner of over 30 years. “He is very comfortable with numbers.” It is this reputation that, so far as I can tell, Mark Penn has written Microtrends to dispel. Unlike most pollsters, Penn never releases his raw numbers, only his analysis. So we must take it on faith that his methodology is rigorous, his polls accurate and his interpretations fair. This book is our first opportunity to observe, at length, how adroitly Penn handles raw data. And the answer is stunning, even to a doubter like me. Mark Penn cannot handle numbers. If this book were turned in as the final to an entry-level statistics class, Penn would not only be failed, but the professor might well retire in shame.

I first flipped through Microtrends while at the YearlyKos convention, and Penn, astonishingly, seemed to comprehend the importance of the loosely connected, grassroots-driven, progressive movement’s flowering. “I suspect the lefty boom will bring a surge in the promotion of sheer creative energy,” Penn writes, “driven by an idea that is at the heart of this book–that small groups of people, sharing common experiences, can increasingly be drawn together to rally for their interests.” I was shocked–Penn was speaking admirably of “lefties,” not trying to recast them as moderates, not trying to write them out of the party? He was endorsing open-source politics, rather than a top-down structure? I had misjudged the man!

I read on. Penn was talking about actual lefties–people who are born left-handed. Increasingly grim, I absorbed the first hard blows of Penn’s interpretative technique: “More lefties,” he enthuses, “could mean more military innovation: Famous military leaders from Charlemagne to Alexander the Great to Julius Caesar to Napoleon–as well as Colin Powell and Norman Schwarzkopf–were left-handed.” He uses the same thunderingly awful logic to argue that we’ll see more art and music greats, more famous criminals, more great comedians, more “executive greatness,” and better tennis and basketball players.

This is what statisticians–or anyone who has taken a statistics class–call a “correlation/causation error.” It is not enough to cherrypick a couple famed military leaders, notice that they’re lefties and assume that something intrinsic to their handedness caused their tactical genius. It is not enough to say that past cultures discouraged left-handedness and use that as a stand-in for discouraging creativity of all sorts. To say that Bill Gates is right-handed does not suggest that a greater proportion of right-handed people would mean more Bill Gateses. For a professional pollster to imply that correlation equals causation is like a firefighter trying to put out flames by tossing a toaster into the blaze–it bespeaks a complete unfamiliarity with the relevant techniques.

"astonishingly"..."astonishingly"..."I was shocked"..."shockingly"
Someone please subdue this feller and take his loaded 'astonishingly' away from him; someone might get hurt. Blunt trauma to the brain.
"Penn would not only be failed..." The insipid book in question, from the evidence offered, most certainly needs a lambasting; 'tis a pity the job was assigned to someone who, astonishingly, writes so shockingly poorly. I wish you would be failed.Posted by rhizome on 2007-11-03 01:41:29

The point made by test100 about margin of error at the extremes is true of real statistics, but we are talking polling here. They can call polling "scientific" all they want, but it lacks the truthiness of statistical analysis of raw data such as home prices, sports stats, income distribution, etc. Polling can't really determine a truthful stat from a false one. There are poll respondents that purposefully answer the questions with lies, and others that don't really pay attention to the question and answer incorrectly. Additionally, polling by telephone is quickly emerging as flawed in the distribution of population due to increasing use of cell phones that aren't called by pollsters.Posted by Jon B on 2007-09-16 03:43:17

That one percent thing is pitiful. Penn knows that this is a representative democracy depending on voting for politicians. One percent can't vote anyone into office and in fact won't even get their one percent displayed in polls on TV. That one percent is utterly powerless so how does Penn attribute influence to that statistical anomaly?Posted by Jon B on 2007-09-15 05:37:29

I was given a galley of this book before it came out, from a local bookseller, who assured me that I would find it fascinating.
Unfortunately, it was utterly idiotic and unreadable, had to skip over pages & pages just to get through. Penn is looking at little tiny stories without any ability whatever to see the larger picture. A real shame.Posted by Mr_Anonymous on 2007-09-15 04:36:21

test100 is right about the margin of error being much smaller for poll results at the extremes, but it doesn't take away from Klein's point about Penn's dubious tactic of investing his 1 percenters with disproportionate influence.
Actually, I don't know why Klein mentioned the margin of error at all, apart from its (incorrect) use as a gotcha moment. It doesn't add to the piece at all, which is strong enough without it.Posted by Eddie visits on 2007-09-15 02:12:58

fyi
Klein is right: 4 is larger than 1. But that has nothing to do with the validity of findings in Sergio BenDixon's polls in the 1 per cent range. Margins of error near the boundaries of 0 and 100 are treated quite differently than the margin of error near the midpoint of 50. (because -3 could never be a possible answer). Klein might want to look that up since it shows such a lack of basic understanding of statistics that it suggests the rest of his observations are about as accurate.Posted by test100 on 2007-09-14 22:31:24

Its a sad day in America if these are the individuals attempting to help shape the direction of this country...
Mr. Penn would be of bigger service helping Microsoft try and position itself against Google or helping General Motors manage the potential upcoming strike against the UAW... LOL
It speaks volumes about the candidacy of Hillary Clinton that Mr. Penn has these little shallow views of Americans...
MicroAmericans... create some soft political fluff and get their votes...
Yup... that is why Mrs. Clinton polls best with lower income older folks... same crowd you can easily merchandise a new shampoo or car to...Posted by Cate on 2007-09-14 13:37:59

Nice article - check out a write up and photos of Mark Penn, and his guests at his recent book launch party in Washington DC.
http://www.washingtonian.com/blogarticles/people/capitalcomment/5199.htmlPosted by bencoffeyclark on 2007-09-14 11:12:37