Conservative bounds on extreme P-values for testing the equality of two probabilities based on very large sample sizes

Abstract

With very large sample sizes the conventional calculations for tests of the equality of two probabilities can lead to very small P-values. In those cases, the large deviation effects make inappropriate the asymptotic normality approximations on which those calculations are based. While reasonable interpretations of the data would tend to reject the hypothesis in those cases, it is desireable to have conservative estimates which don’t underestimate the P-value. The calculation of such estimates is presented here.