Electronic Journal of Statistics

Some non-asymptotic properties of parametric bootstrap P-values in discrete models

Chris J. Lloyd

Full-text: Open access

Abstract

For discrete data especially, standard P-values can misreport the true significance, even for moderately large sample sizes. The bootstrap P-value is the exact tail probability of an appropriate test statistic, calculated assuming the nuisance parameter equals the null maximum likelihood (ML) estimate. For basic discrete models and standard test statistics, bootstrap P-values have been found to be extremely close to uniformly distributed under the null ([1]). Detailed numerical results reported there suggest that this phenomenon is not explained by asymptotics. In this paper, we identify several desirable non-asymptotic properties of bootstrap P-values and provide arguments for why bootstrap P-values are so close to exact. The most important of these is that bootstrap will correct ‘incorrect’ ordering of the sample space and that this leads to a more pivotal distribution. Most of these arguments only hold for discrete models and when the null ML estimate is used.

Article information

Source
Electron. J. Statist., Volume 6 (2012), 2449-2462.

Dates
First available in Project Euclid: 4 January 2013

Permanent link to this document
https://projecteuclid.org/euclid.ejs/1357307945

Digital Object Identifier
doi:10.1214/12-EJS751

Mathematical Reviews number (MathSciNet)
MR3020271

Zentralblatt MATH identifier
1295.62015

Subjects
Primary: 62F03: Hypothesis testing
Secondary: 62F05: Asymptotic properties of tests

Keywords
Bootstrap exact test exact P-value nuisance parameters size accuracy

Citation

Lloyd, Chris J. Some non-asymptotic properties of parametric bootstrap P-values in discrete models. Electron. J. Statist. 6 (2012), 2449--2462. doi:10.1214/12-EJS751. https://projecteuclid.org/euclid.ejs/1357307945


Export citation

References

  • [1] Lloyd, C.J. (2013). A numerical investigation of the accuracy of parametric bootstrap for discrete data., Comp. Statist. Data Anal. 57. To appear.
  • [2] Cressie, N. and Read, T.R.C. (1984). Multinomial goodness-of-fit tests., J. Roy. Statist. Soc. B 46 157–214.
  • [3] Perkins, W., Tygert, M. and Ward, R. (2011). Computing the confidence levels for a root-mean-square test of goodness-of-fit, Appl. Math. Comput. 217 9072–9084.
  • [4] Beran, R. (1988). Prepivoting test statistics: a bootstrap view of asymptotic refinements., J. Amer. Statist. Assoc. 83 687–697.
  • [5] Rohmel, J. and Mansmann, U. (1999). Unconditional non-asymptotic one-sided tests for independent binomial proportions when the interest lies in showing non-inferiority or superiority., Biometrical Journal 41 149–170.
  • [6] DiCiccio, T.J., Martin, M.A. and Stern, S.E. (1984). Simple and accurate one-sided inference from signed roots of likelihood ratios., Canad. J. Statist. 29 167–76.
  • [7] DiCiccio, T.J. and Stern, S.E. (1994). Constructing approximate standard normal pivots from signed roots of adjusted likelihood ratio statistics., Scand. J. Statist. 21 447–460.
  • [8] Fraser, D.A.S and Rousseau, J. (2008.) Studentization and deriving accurate P-values., Biometrika 95 1–16.
  • [9] Freeman, G.H. and Halton, J.H. (1951). Note on exact treatment of contingency, goodness of fit and other problems of significance., Biometrika 38 141–149.
  • [10] Lee, S.M.S. and Young, G.A. (2005). Parametric bootstrapping with nuisance parameters., Stat. Prob. Letters 71 143–153.
  • [11] Lloyd, C.J. (2008). Exact P-values for discrete models obtained by estimation and maximisation., Austral. and New Zealand J. Statist. 50 329–346.
  • [12] Lloyd, C.J. (2012). Computing Highly Accurate or Exact P-values using Importance Sampling., Comp. Statist. Data Anal. 56 1784–1794.
  • [13] Berger, R.L. and Boos, D.D. (1994). P values maximised over a confidence set for the nuisance parameter., J. Amer. Statist. Assoc. 89 1012–1016.