Articles and Analysis

Earlier this morning, I posted a response to a comment
on yesterday's entry from reader Danielle Clark that on reflection is worth
discussing further on the main page. Her comments concern the ways that
pollsters handle "undecided" voters:

For those wondering why Obama's poll numbers seem
surprisingly low, check the raw results as opposed to the press releases. If
the raw results aren't posted on the polling company's site, there's probably a
reason for that. ;-)

Awhile back I learned it's considered acceptable to
drop, re-allocate or report "Don't Know" answers - it all depends on
the polling company's choice, and they don't tend to disclose which they did.
This can cause misleading results, especially at this point when there are
still a lot of people who don't know enough about Obama (and other candidates)
to want to commit to one.

For example, a poll may find 5% support Obama, 10% Clinton, 10% others, and
75% don't know. It's considered acceptable to report this as 20% support Obama,
40% Clinton,
and 40% others - which implies everyone asked had an opinion of some sort.

Another reason to insist on seeing the raw data is
to know what questions were asked before the question one's interested in -
those can clearly shape the answer.

She also linked to a useful paper on "Interpreting Polls " put out by a
parliamentary librarian in Australia.
Aside from our regular commentary here, I would also recommend the resources available on the new web page of the
American Association for Public Opinion Research (AAPOR).

Danielle was right to recommend examining the verbatim text of survey
questions and the way pollsters calculate their results. Where available, we
include links to the most complete report of results in our "poll update" posts
and in the tables that appear below the charts in our Polls section. The Polling Report also provides verbatim text for poll questions. Follow those links to
see the full results and verbatim text when available.

However, we need to be careful to distinguish between tables or references
in news stories that omits the results for the undecided category and
those that reallocate undecided voters as Danielle describes.

In the US,
virtually all polls are now reporting trial heat results based on calculations
that include the undecided or "don't know" category. Some pollsters
will put out "projections" based on their final poll before the election that
use various methods to reallocate undecided voters (the Polling Report has a
handy set of tables
showing both "vote projections" and "final trial heats" from 2004 side by
side). But I know of no pollsters putting out any such projections right now.

One way to check, if the undecided category is omitted, is to add up the
results to each candidate. If they total 100%, then the results have almost
certainly been recalculated.

More to the point, all of the results posted and plotted in the
charts and tables in the Polls
section of Pollster.com -- including the numbers discussed in yesterday's post
-- are based on calculations that include the undecided category.

It is also important to distinguish between the undecided category on a trial heat question and separate questions that probe how
certain or "decided" voters say they are. The undecided category tends to
understate the real potential for change in a political contest. In recent
weeks have written about questions asked on recent LA
Times/Bloomberg and CNN/WMUR/UNH
polls showing huge number of voters in Iowa and New Hampshire that are still less
than certain about their choice.

Danielle may be thinking about computations that take those "uncertain"
voters into account that Obama himself has been touting on the campaign trail. For
example, NBC's First Read
reported the following recent comments by Barack Obama at a rally in New Hampshire:

[Obama] also referred to a recent poll that had
Sen. Clinton in the lead by twenty points, and said that it meant nothing
besides what percentage of people were supporting which candidate. "Twenty
percent of voters are with Sen. Clinton, 10 percent are with me, Edwards has
about 5 percent and the rest of the candidates have less than that."

Citing that same poll, Obama said the most important fact overlooked by the
pundits were the number of undecideds still up for grabs. "There are 55
percent of voters who are undecided. That's 55 to 60 percent of the people who
are waiting to hear from you," he told the crowd of campaign volunteers
made up of students from colleges in Massachusetts
and across New Hampshire
as well as local volunteers.

Obama was probably referring to the recent CNN/WMUR/UNH
poll that gave Clinton a 22 point lead over Obama (with only 9%
"undecided"), but also showed 55% of New Hampshire likely Democratic primary
voters saying on a separate question that they are "still trying to decide"
which candidate to support. If I set aside the uncertain 55% and recalculate
vote preference numbers among those who say they are "definitely decided" or
"leaning," I get 23% for Clinton,
10% for Obama and 6% for Edwards.

Post a comment

Name:

Email Address:

URL:

Comments: (you may use HTML tags for style)

Please be patient while your comment posts - sometimes it takes a minute or two. To check your comment, please wait 60 seconds and click your browser's refresh button. Note that comments with three or more hyperlinks will be held for approval.