Testimony Proffered to New York Supreme Court
Judge Ira Warshawsky Regarding the November 2010 Senate Contest in NY Senate District 7

Philip B. Stark, 4 December 2010

I prepared the following testimony for a hearing before Judge Ira Warshawsky (New York Supreme Court)
on 4 December 2010.
Some portion was read into the record, but not the entire statement.
I have since discovered that I overlooked one error that the audit had found, which favored Mr. Johnson; I have
corrected this statement to take that error into account.

Judge Warshawsky denied the request to expand the audit or have a full hand count.
The plaintiff appealed; the appellate court upheld Judge Warshawsky's decision, 4–0.
The Court of Appeals upheld the decision as well, with an unsound statistical argument.

The 3% audit gives very little statistical confidence that a full hand count of the
ballots would show Mr. Martins to be the winner.
The audit results would not be surprising even if a full hand count would show
Mr. Johnson to be the winner.

Because the audit examined only 7 machines, there is a substantial possibility that
the machine with the largest error was not one of the machines that was audited.
Indeed, there's a 97% chance that auditing 7 of 249 machines won't check the machine
with the largest error.

An average of less than two errors per machine could account for the apparent
margin of about 450 votes.
An average of one error per 200 ballots could account for the apparent margin.

In my experience, this level of error in precinct-count optically scanned ballots would
not be surprising. And it is consistent with the errors the audit did find, within the
statistical variability expected from "the luck of the draw."

The potential for error in this contest is large: In total, the 242 unaudited machines
could hold enough error to account for the apparent margin 186 times over.
Sixty-six of the 242 unaudited machines could individually hold enough error to account for
the apparent margin.

There is a substantial chance that a 3% or 8% audit would find little or no error
even if Mr. Johnson is the true winner.

For instance, if 30 of the 249 machines have errors of 15 votes or more--enough to account
for the apparent margin--the chance that the 3% audit would have found any of those
machines is under 60%.

If 20 of the 249 machines have errors of 23 votes or more--enough to account for the
apparent margin--the chance that the 3% audit would have found any of those machines is
under 45%.

If 20 of the 242 unaudited machines have errors of 23 votes or more (enough to
account for the apparent margin) and an additional 5% of the machines are audited, the
chance that the additional audit would find any of those 20 is under 69%.

Because the margin is so small compared to the possible errors, a very large percentage
of machines needs to be audited to give strong evidence that Mr. Martins is indeed the
winner.
3% is not sufficient.
8% is not sufficient.
To have 90% statistical confidence that Mr. Martins won requires auditing a minimum of
90% of the machines selected randomly: an additional 218 machines.

This is true if the audit finds that those 218 machines have counted perfectly.
If the audit of those 218 machines found many errors, still more machines would have
to be audited.

[1] One of the seven audited machines had two errors that favored Mr. Martins, one had one error that favored Mr. Martins, and one had one error
that favored Mr. Johnson.