There are many sources of error in counting votes: the apparent winner might not be the rightful winner. Hand tallies of the votes in a random sample of precincts can be used to test the hypothesis that a full manual recount would find a different outcome. This paper develops a conservative sequential test based on the vote-counting errors found in a hand tally of a simple or stratified random sample of precincts. The procedure includes a natural escalation: If the hypothesis that the apparent outcome is incorrect is not rejected at stage s, more precincts are audited. Eventually, either the hypothesis is rejected—and the apparent outcome is confirmed—or all precincts have been audited and the true outcome is known. The test uses a priori bounds on the overstatement of the margin that could result from error in each precinct. Such bounds can be derived from the reported counts in each precinct and upper bounds on the number of votes cast in each precinct. The test allows errors in different precincts to be treated differently to reflect voting technology or precinct sizes. It is not optimal, but it is conservative: the chance of erroneously confirming the outcome of a contest if a full manual recount would show a different outcome is no larger than the nominal significance level. The approach also gives a conservative P-value for the hypothesis that a full manual recount would find a different outcome, given the errors found in a fixed size sample. This is illustrated with two contests from November, 2006: the U.S. Senate race in Minnesota and a school board race for the Sausalito Marin City School District in California, a small contest in which voters could vote for up to three candidates.
References
Aslam, J. A., Popa, R. A. and Rivest, R. L. (2007). On auditing elections when precincts have different sizes. Available at people.csail.mit.edu/rivest/AslamPopaRivest-OnAuditingElectionsWhenPrecinctsHaveDifferentSizes.pdf.
Benjamini, Y. and Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. J. Roy. Statist. Soc. Ser. B 57 289–300.
Bjornlund, E. C. (2004). Beyond Free and Fair: Monitoring Elections and Building Democracy. Woodrow Wilson Center Press, Washington, DC.
Dopp, K. and Stenger, F. (2006). The election integrity audit. Available at uscountvotes.org/ucvInfo/release/ElectionIntegrityAudit-release.pdf.
Elections: Federal efforts to improve security and reliability of electronic voting systems are under way, but key activities need to be completed (2005). Technical Report GAO-05-956, U.S. Government Accountability Office, Washington, DC.
Elections: The nation’s evolving election system as reflected in the November 2004 general election (2006). Technical Report GAO-06-450, U.S. Government Accountability Office, Washington, DC.
Estok, M., Nevitte, N. and Cowan, G. (2002). The Quick Count and Election Observation. National Democratic Institute for International Affairs, Washington, DC.
Halvorson, M. and Wolff, L. (2007). Report and analysis of the 2006 post-election audit of Minnesotas voting systems. Available at ceimn.org/files/CEIMNAuditReport2006.pdf.
Hite, R. C. (2007). Elections: All levels of government are needed to address electronic voting system challenges. Technical Report GAO-07-714T, U.S. Government Accountability Office, Washington, DC.
Hoeffding, W. (1956). On the distribution of the number of successes in independent trials. Ann. Math. Statist. 27 713–721.
Jefferson, D., Alexander, K., Ginnold, E., Lehmkuhl, A., Midstokke, K. and Stark, P. B. (2007). Post election audit standards report–evaluation of audit sampling models and options for strengthening Californias manual count. Available at www.sos.ca.gov/elections/peas/final_peaswg_report.pdf.
McCarthy, J., Stanislevic, H., Lindeman, M., Ash, A., Addona, V. and Batcher, M. (2008). Percentage based vs. statistical-power-based vote tabulation auditing. The American Statistician 62 11–16.
Norden, L., Burstein, A., Hall, J. L. and Chen, M. (2007). Post-election audits: Restoring trust in elections. Technical report, Brennan Center for Justice, New York University and Samuelson Law, Technology & Public Policy Clinic at University of California, Berkeley School of Law (Boalt Hall), New York.
National Association of Secretaries of State (2007). Post election audit procedures by state. Available at nass.org/index.php?option=com_docman&task=doc_download&gid=54.
Panel on Nonstandard Mixtures of Distributions (1989). Statistical models and analysis in auditing: Panel on nonstandard mixtures of distributions. Statist. Sci. 4 2–33.
Rivest, R. L. (2006). On estimating the size of a statistical audit. Available at people.csail.mit.edu/rivest/Rivest-OnEstimatingTheSizeOfAStatisticalAudit.pdf.
Saltman, R. G. (1975). Effective use of computing technology in vote-tallying. Technical Report NBSIR 75-687, National Bureau of Standards, Washington, DC.
Siegmund, D. (1985). Sequential Analysis: Tests and Confidence Intervals. Springer, New York.
Stanislevic, H. (2006). Random auditing of e-voting systems: How much is enough? Available at www.votetrustusa.org/pdfs/VTTF/EVEPAuditing.pdf.
Stark, P. B. (2008a). Election audits by sampling with probability proportional to an error bound: Dealing with discrepancies. Available at statistics.berkeley.edu/~stark/Preprints/ppebwrwd08.pdf.
Stark, P. B. (2008b). A sharper discrepancy measure for post-election audits. Ann. Appl. Statist. To appear.
Verified Voting Foundation (2007). Manual audit requirements. Available at www.verifiedvoting.org/downloads/stateaudits1007.pdf.
Wald, A. (2004). Sequential Analysis. Dover, Mineola, NY.
