THE 2004 ELECTIONS: THE ELECTORATE -- THE POLLING; Survey Experts Cite Problems With Data and Interpretation

By JIM RUTENBERG; Christopher Drew and Jodi Wilgoren contributed reporting for this article.

Published: November 4, 2004

The $10 million, state-of-the-art system the major television networks built to project accurately this year's Election Day vote was supposed to fix the problems that led to their misbegotten calls of 2000.

But only hours after the returns began flowing in on Tuesday, as early indications that Senator John Kerry would beat President Bush were disproved, the recriminations began. And yesterday they only intensified.

''My initial sense of this is that the exit polls were all wrong,'' Mary Beth Cahill, Mr. Kerry's campaign manager, said Wednesday afternoon in a sentiment that was echoed by officials from both presidential campaigns and some media critics. The critiques seemed to surprise some network officials, who said they were generally satisfied with the system, adding that the surveys of voters leaving the polls did not lead them to make any erroneous calls on Tuesday night.

But they did acknowledge that even though they are trained to view carefully data that are ultimately estimates, the perception that Mr. Kerry was headed for victory may have crept into their reporting, albeit subtly. More important, the results of the voter surveys -- once the sole provenance of the news media and political insiders -- found their way to various Web sites and then millions of visitors, many of whom accepted preliminary data as gospel.

As Mr. Bush's brother, Gov. Jeb Bush of Florida, put it in a briefing with reporters yesterday, ''The way this works is that someone leaks the exit polls to somebody.'' He went on, ''within 20 minutes, with BlackBerrys, everybody's got the exit polls.''

Amid the sniping and assessment yesterday, two questions prevailed: Was it that the survey data was wrong? Or were people unused to analyzing such data just misinterpreting it?

Pollsters, campaign strategists from both sides and news executives said in interviews that that the answer is, probably, a little bit of both.

From the Drudge Report to Wonkette to DailyKos, Web sites were often reporting the survey data in its pure form -- Kerry 50; Bush 49 in Ohio -- without explaining in depth important caveats, such as that the results fell within a margin of error. So in Ohio, for instance, the first report fell with a margin of error of plus or minus five percentage points, meaning a candidate would have to have a lead of more than 10 points to have a statistically significant lead.

Those error margins shrunk as the samples grew larger, leaving less room for error.

Exit polls, like any other polls, are devised to be accurate when they are finished, not when they are partly finished, said Mark Mellman, a pollster for Mr. Kerry.

''If you have a poll a third of the way done,'' Mr. Mellman said, ''why would you expect it to be accurate?''

In fact, all of the polls early in the day showing Mr. Kerry ahead in important swing states were within their margins of error, and the same was true for a national survey showing Mr. Kerry ahead of Mr. Bush.

But early on, both political camps noticed problems. For one, an early version of the national poll showing Mr. Kerry ahead of Mr. Bush was taking a sample of voters that was disproportionately female, possibly skewing it toward Mr. Kerry.

By late afternoon, the national survey showing Mr. Kerry winning the popular vote by three percentage points became statistically significant, falling outside the survey's margin of error.

But news producers said Joe Lenski and Warren Mitofsky, the men who devised and ran the system for a consortium of ABC, CNN, CBS, NBC, Fox News Channel and The Associated Press, warned them of some of the problems in the afternoon.

Mr. Lenski said in an interview yesterday that it was possible that more Democrats and more women were voting earlier, perhaps skewing the data in the afternoon. But, he said, by the end of the night the system's polling data basically tracked with the actual results.

''Sophisticated users of this data know the limitations of partial survey results,'' he said.

Yet some members said they would take a hard look at the system.

''There were significant problems with the exit poll, and it's going to take a while to get to what the problems were,'' said John Gorman, a pollster working for Fox News Channel on Tuesday night.

Graph: ''Voter Turnout''
Number of votes cast for president as a percentage of the voting-age population eligible to vote: