Just Say "No" to Pollsters?

May 18, 2003

TV commentator, author, and columnist Arianna Huffington thinks politicians are hooked on polls in ways that are dangerous for democracy. She has a solution. "When pollsters call, just refuse to answer any questions," she urges voters on her Web site, www.ariannaonline.com. "Once enough people join us, poll results will become useless, and our leaders will be forced to lead."

So fireworks were bound to explode on May 16, when Huffington, the self-described co-founder of the Partnership for a Poll-Free America, agreed to address the annual meeting in Nashville of the American Association for Public Opinion Research (www.aapor.org), the nation's leading organization for poll researchers. By the evening's end, Huffington was on the defensive, dodging accusations that she had her facts wrong and protesting that she had been misunderstood.

It was an exhilarating moment for the researchers, who fear that Huffington's "crusade," as she calls it, will undermine their ability to gather information from the public on all kinds of topics -- not only on voter preferences but on how many people are unemployed, how many children haven't been vaccinated, and whether old people understand the eligibility for health benefits. (Disclosure: In addition to being BusinessWeek's Economics editor, I'm its liaison to survey organizations. Also, I've attended several meetings of the AAPOR, and my wife is a member.)

POLS' ABUSES. Huffington's main criticism is that excessive attention to polls has turned politicians from courageous leaders into pandering marketers. Using polls, candidates and incumbents continuously fine-tune their platforms and messages to make people like them, instead of deciding what they think is right and trying to rally people to follow.

The pollsters' response? All that may be true, but it's not their fault if their results are misused. It's important to know where the public stands on the issues of the day, said Richard Morin, a writer for The Washington Post who was one of three people assigned to engage Huffington in a discussion for the audience.

Huffington alleges in her newspaper columns that polls are getting increasingly inaccurate. In fact, said Morin, the average error of polls concerning the two major candidates in the 2000 elections was just 1.1 percentage points, significantly lower than the typical margin of error 20 or 30 years ago.

STATISTICAL DEAD HEAT. The pundit claims that in the 2000 Presidential election, in which Al Gore won the popular vote, 80% of pollsters falsely picked George W. Bush to win more of the popular vote than Gore or to tie. In fact, say the researchers, even though most of the polls had Bush slightly ahead, the pollsters actually considered the race a statistical dead heat -- that is, Bush's tiny advantage was less than the polls' stated margin of error.

Is accuracy being degraded by the increasing rate of "nonresponse" -- in which phone calls aren't answered or people do answer and refuse to participate? That's what Huffington charges. It's a reasonable criticism, assuming that the people who do answer surveys are different from the ones who don't. But several papers at the conference presented surprising evidence that for a broad range of nonresponse rates -- say, anywhere from 30% to 70% -- more nonresponses don't worsen accuracy.

Finally, Huffington contends that polls were partially responsible for the tragedy of September 11, 2001. She says they showed that the public didn't consider terrorism a major threat, so poll-driven politicians ignored security. But Morin produced a series of polls conducted before September 11 in which the public did express major concern about terrorism -- in contrast to only three "fleeting" mentions of terrorism he found in Huffington's own prodigious output of columns.

END OF ALL POLLS? Not all of Huffington's criticisms were rejected as out of hand. The AAPOR has begun urging its members to publish the rate of nonresponse to their surveys, a step Huffington advocates. And like her, the group is on-record opposing so-called "commissioned polls" that are engineered to show public support for an organization's agenda.

What most galled the researchers, however, was Huffington's call for shutting down polling entirely. The critic said she was against only polls that were designed to help politicians tailor their messages. But when the researchers asked her to clarify that distinction in her columns and on her Web site, she wouldn't commit to doing so. On that score, the evening ended in a draw. By Peter Coy in Nashville