class: blueBack ## Auditing and Evidence ### Philip B. Stark
Department of Statistics
University of California, Berkeley ### Election Verification Network Annual Conference ### New Orleans, LA
4–6 March 2015 --- ### .blue[Pop Quiz:] -- #### .blue[Why do we want paper?] -- #### .green[Voter verifiability] + most voters can check directly whether the paper reflects their choices + the rest can using some form of assistive technology -- #### .green[Audit trail] + paper is durable, tamper-evident, trackable, manageable by humans -- #### .green[Auditability] + if we keep track of the paper, we can check the counting process by manually inspecting ballots -- ### .blue[Evidence!] (but you still have to look) --- ### .blue[Evidence-based elections] + Laws and regulations should ask LEOs to give strong evidence that election results are right (or fess up) -- + Following rules isn't enough -- + If you want to know whether surgery was successful, it's not enough to ask whether the surgeon washed her hands: gotta look at the patient -- + That's what auditing is about. --- ### .blue[Audits] + Common claim: "all we care about is whether the machines are functioning properly." -- + What does "properly" mean? -- + .red[*Always* some rate of error. Not a "yes or no" question.] -- + .blue[Minimum requirement: machines should function well enough to tell who really won!] -- + Spot-checking machines doesn't tell us that. -- + The minimum acceptable amount of checking depends on the margin + small margins mean a small rate of errors could have altered the outcome: need more checking --- ### .blue[It's the morning after the election.] -- ### .red[Do you know where your ballots are?] -- Audits are a farce unless there is verifiably good chain of custody, ballot accounting, and so on. -- ### .red[Providing 'ballot manifests' should be mandatory, now.] -- ### Needed for *any* auditing. --- ### .blue[Risk limiting audits] -- + Key idea: a good audit should correct the outcome if the outcome is wrong -- + How do we do that? -- + Keep collecting evidence (inspecting ballots manually) until either + there's strong evidence that a full count would confirm the outcome, or + you've done a full count and know the real outcome --- ### .blue[Approach 1] + .green[Ballot-polling audits] + only require ballots (and ballot manifest). No special equipment, or set-up + like an exit poll, but of ballots + cheap unless the margin is really small + can just use dice and a pencil and paper --- ### .blue[Approach 2] + .green[Comparison audits] + requires ballots, ballot manifest, **and** voting system has to report subtotals + have to be able to retrieve the ballots corresponding to each subtotal + lots of set-up for current equipment: "data plumbing" is poor + efficiency depends on number of ballots in subtotals: + best if equipment gives individual CVRs: *ballot-level audits* + no commercial system currently does + for precinct subtotals, inefficient compared to ballot-polling --- ### .blue[How hard is it?] + Simple web tools for both types of audit. The calculations are easy. + http://www.stat.berkeley.edu/~stark/Vote/auditTools.htm + http://www.stat.berkeley.edu/~stark/Vote/ballotPollTools.htm -- + Better-designed voting systems could make ballot-level comparison audits **very fast and easy** --- ### .blue[Pilot Audits] #### California + Alameda, Humboldt, Madera, Marin, Merced, Monterey, Napa, Orange, San Luis Obispo, Santa Cruz, Stanislaus, Ventura, Yolo -- #### Colorado + Arapahoe, Boulder -- #### Ohio + Cuyahoga --- ### .blue[What can/should states do at "election speed"] -- + Move to paper if they haven't already -- + Establish good rules for keeping track of paper, chain of custody, ballot reconciliation, etc. -- + Report how many ballots were cast, and how they are organized ("ballot manifest") -- + Start ballot-polling RLAs for statewide contests -- + Use RLAs as principled substitute for automatic recounts (fairer and generally less work) -- + As equipment is replaced, make sure it reports individual CVRs, tied to the corresponding paper ballot -- + Once a jurisdiction can match paper to CVRs, start ballot-level comparison RLAs for local contests --- ### .blue[Election Integrity Commandments] 1. Thou shalt have paper; paper shall be thy medium of record. 1. Thy paper shall be accessible to all voters. 1. All voters shall have the opportunity to check the paper before casting their votes, and a divine right to start from scratch if the visage of the paper pleaseth them not. 1. Thou shalt keep thy paper wholly, cherish it, and hold it inviolate. 1. Thou shalt prove that thou hast kept track of thy paper, through ballot accounting, chain of custody logs, and their kin. 1. Thou shalt audit results in a manner that will correct the majority of incorrect results, by making use of the casting of lots. 1. Thine eyes and the eyes of the public on thy paper shall take precedence over any technologickal magicks that would speak for the paper.