Some non-asymptotic properties of parametric bootstrap P-values in discrete models

Abstract

For discrete data especially, standard P-values can misreport the true significance, even for moderately large sample sizes. The bootstrap P-value is the exact tail probability of an appropriate test statistic, calculated assuming the nuisance parameter equals the null maximum likelihood (ML) estimate. For basic discrete models and standard test statistics, bootstrap P-values have been found to be extremely close to uniformly distributed under the null ([1]). Detailed numerical results reported there suggest that this phenomenon is not explained by asymptotics. In this paper, we identify several desirable non-asymptotic properties of bootstrap P-values and provide arguments for why bootstrap P-values are so close to exact. The most important of these is that bootstrap will correct ‘incorrect’ ordering of the sample space and that this leads to a more pivotal distribution. Most of these arguments only hold for discrete models and when the null ML estimate is used.