Risk-limiting post-election audits
                               
                               Philip Stark
                     Department of Statistics, UC Berkeley

Free, fair and accurate elections are the cornerstone of democracy.
Post-election audits--which compare reported totals with hand counts
of a trustworthy audit trail in a random sample of batches--can limit
the risk of certifying an incorrect electoral outcome.  The risk is
the maximum chance the audit stops short of a full hand count when a
full hand count would show that the outcome is wrong.  Post-election
audits can measure and improve the accuracy of elections, increase
transparency and bolster public confidence.  They are at the
intersection of law, public policy, public administration, public
accountability and statistics.

Several states have post-election audits. Federal legislation
requiring audits has been introduced several times.  But no existing
or proposed audit law controls the risk of certifying an incorrect
outcome.  Risk-limiting audits can be built as sequential tests: Data
are collected.  If they give strong evidence that the outcome is
right, the audit stops. Otherwise, more data are collected.
Eventually, the audit stops or there has been a full hand count. The
strength of the evidence can be quantified using the maximum P-value
of the hypothesis that the outcome is wrong given the audit data; that
computation depends on the sampling scheme, the choice of test
statistics, etc.  Multiplicity can be dealt with in several ways.

There have been four risk-limiting audits, all in California in 2008:
Marin County (a small measure in February requiring a supermajority
and a county-wide measure in November), Santa Cruz County (County
Supervisor, District 1, November), and Yolo County (bond measure in
November).  We designed these audits, which were conducted in
collaboration with elections officials officials in the counties.
Several sampling techniques were tested.  The audits ensured at least
a 75% chance of a full hand count if that would change the outcome.
All four outcomes were confirmed without a full hand count.

Many lessons were learned. Clear, precise and timely communication
between the auditors and the elections officials is key.  Optimality
is a low priority compared with simplicity.  The biggest barrier to
wide-scale risk-limiting audits is the inability of current election
management systems (vote tabulation systems) to export data in a
useful, machine-readable format.  Adopting standard terminology and
data formats would be extremely helpful.

Post-election auditing has interesting connections to financial
auditing and to statistical topics such as nonparametric inference
about the mean of a nonnegative random variable, sequential testing,
and the false discovery rate.

This work is joint in part with Luke Miratrix, Joe Hall and Mike Higgins.