Ever since its introduction, the bootstrap has provided both a powerful set of
solutions for practical statisticians, and a rich source of theoretical and methodological
problems for statistics. In this article, some recent
developments in bootstrap methodology are reviewed and discussed.
After a brief introduction
to the bootstrap, we consider the following topics at varying levels of detail: the use of
bootstrapping for highly accurate parametric inference; theoretical properties of nonparametric
bootstrapping with unequal probabilities; subsampling and the m out
of n bootstrap;
bootstrap failures and remedies for superefficient estimators;
recent topics in significance
testing; bootstrap improvements of unstable classifiers and
resampling for dependent data.
The treatment is telegraphic rather than exhaustive.
References
Baggerly, K. A. (1998). Empirical likelihood as a goodness-of-fit measure. Biometrika 85 535--547.
Barndorff-Nielsen, O. E. (1980). Conditionality resolutions. Biometrika 67 293--310.
Mathematical Reviews (MathSciNet):
MR581727
Barndorff-Nielsen, O. E. (1983). On a formula for the distribution of the maximum likelihood estimator. Biometrika 70 343--365.
Mathematical Reviews (MathSciNet):
MR712023
Barndorff-Nielsen, O. E. (1986). Inference on full or partial parameters based on the standardized signed log likelihood ratio. Biometrika 73 307--322.
Mathematical Reviews (MathSciNet):
MR855891
Barndorff-Nielsen, O. E. and Cox, D. R. (1979). Edgeworth and saddle-point approximations with statistical applications (with discussion). J. Roy. Statist. Soc. Ser. B 41 279--312.
Mathematical Reviews (MathSciNet):
MR557595
Barndorff-Nielsen, O. E. and Cox, D. R. (1994). Inference and Asymptotics. Chapman and Hall, London.
Beran R. J. (1986). Simulated power functions. Ann. Statist. 14 151--173.
Mathematical Reviews (MathSciNet):
MR829560
Beran, R. J. (1987). Prepivoting to reduce level error of confidence sets. Biometrika 74 457--468.
Mathematical Reviews (MathSciNet):
MR909351
Beran, R. J. (1988). Prepivoting test statistics: A bootstrap view of asymptotic refinements. J. Amer. Statist. Assoc. 83 687--697.
Mathematical Reviews (MathSciNet):
MR963796
Beran, R. J. (1995). Stein confidence sets and the bootstrap. Statist. Sinica 5 109--127.
Beran, R. J. (1997). Diagnosing bootstrap success. Ann. Inst. Statist. Math. 49 l--24.
Beran, R. J. (2003). The impact of the bootstrap on statistical algorithms and theory. Statist. Sci. 18 175--184 (this issue).
Bickel, P. J. and Freedman, D. A. (1981). Some asymptotic theory for the bootstrap. Ann. Statist. 9 1196--1217.
Mathematical Reviews (MathSciNet):
MR630103
Bickel, P. J. and Ghosh, J. K. (1990). A decomposition for the likelihood ratio statistic and the Bartlett correction--- a Bayesian argument. Ann. Statist. 18 1070--1090.
Bickel, P. J., Götze, F. and van Zwet, W. R. (1997). Resampling fewer than $n$ observations: Gains, losses, and remedies for losses. Statist. Sinica 7 1--32.
Booth, J. G. and Hobert, J. P. (1998). Standard errors of prediction in generalized linear mixed models. J. Amer. Statist. Assoc. 93 262--272.
Brazzale, A. R. (2000). Practical small-sample parametric inference. Ph.D. dissertation, Dept. Mathematics, Swiss Federal Institute of Technology, Lausanne.
Breiman, L. (1996a). Heuristics of instability and stabilization in model selection. Ann. Statist. 24 2350--2383.
Breiman, L. (1996b). Bagging predictors. Machine Learning 24 123--140.
Bretagnolle, J. (1983). Lois limites du bootstrap de certaines fonctionelles. Ann. Inst. H. Poincaré Probab. Statist. 19 281--296.
Mathematical Reviews (MathSciNet):
MR725561
Brumback, B. A. and Rice, J. A. (1998). Smoothing spline models for the analysis of nested and crossed samples of curves (with discussion). J. Amer. Statist. Assoc. 93 961--994.
Bühlmann, P. (2002a). Bootstraps for time series. Statist. Sci. 17 52--72.
Bühlmann, P. (2002b). Sieve bootstrap with variable-length Markov chains for stationary categorical time series (with discussion). J. Amer. Statist. Assoc. 97 443--471.
Bühlmann, P. and Yu, B. (2002). Analyzing bagging. Ann. Statist. 30 927--961.
Canty, A. J., Davison, A. C., Hinkley, D. V. and Ventura, V. (2002). Bootstrap diagnostics. Preprint, Institute of Mathematics, Swiss Federal Institute of Technology, Lausanne.
Carpenter, J. (1999). Test inversion bootstrap confidence intervals. J. R. Stat. Soc. Ser. B Stat. Methodol. 61 159--172.
Corcoran, S. A. (1998). Bartlett adjustment of empirical discrepancy statistics. Biometrika 85 967--972.
Cox, D. R. (1980). Local ancillarity. Biometrika 67 279--286.
Mathematical Reviews (MathSciNet):
MR581725
Davison, A. C. (2003). Statistical Models. Cambridge Univ. Press.
Davison, A. C. and Hinkley, D. V. (1997). Bootstrap Methods and Their Application. Cambridge Univ. Press.
Delgado, M. A. and González Manteiga, W. (2001). Significance testing in nonparametric regression based on the bootstrap. Ann. Statist. 29 1469--1507.
DiCiccio, T. J. and Efron, B. (1992). More accurate confidence intervals in exponential families. Biometrika 79 231--245.
DiCiccio, T. J. and Efron, B. (1996). Bootstrap confidence intervals (with discussion). Statist. Sci. 11 189--228.
DiCiccio, T. J., Martin, M. A. and Stern, S. E. (2001). Simple and accurate one-sided inference from signed roots of likelihood ratios. Canad. J. Statist. 29 67--76.
DiCiccio, T. J. and Romano, J. P. (1990). Nonparametric confidence limits by resampling methods and least favorable families. Internat. Statist. Rev. 58 59--76.
DiCiccio, T. J. and Romano, J. P. (1995). On bootstrap procedures for second-order accurate confidence limits in parametric models. Statist. Sinica 5 141--160.
Durbin, J. (1980). Approximations for densities of sufficient estimators. Biometrika 67 311--333.
Mathematical Reviews (MathSciNet):
MR581728
Efron, B. (1979). Bootstrap methods: Another look at the jackknife. Ann. Statist. 7 1--26.
Mathematical Reviews (MathSciNet):
MR515681
Efron, B. (1983). Estimating the error rate of a prediction rule: Improvement on cross-validation. J. Amer. Statist. Assoc. 78 316--331.
Mathematical Reviews (MathSciNet):
MR711106
Efron, B. (1986). How biased is the apparent error rate of a prediction rule? J. Amer. Statist. Assoc. 81 461--470.
Mathematical Reviews (MathSciNet):
MR845884
Efron, B. (1987). Better bootstrap confidence intervals (with discussion). J. Amer. Statist. Assoc. 82 171--200.
Mathematical Reviews (MathSciNet):
MR883345
Efron, B. and Hinkley, D. V. (1978). Assessing the accuracy of the maximum likelihood estimator: Observed versus expected Fisher information (with discussion). Biometrika 65 457--487.
Mathematical Reviews (MathSciNet):
MR521817
Efron, B. and Tibshirani, R. J. (1993). An Introduction to the Bootstrap. Chapman and Hall, New York.
Efron, B. and Tibshirani, R. J. (1997). Improvements on cross-validation: The $.632+$ bootstrap method. J. Amer. Statist. Assoc. 92 548--560.
Efron, B. and Tibshirani, R. J. (1998). The problem of regions. Ann. Statist. 26 1687--1718.
Fan, J. and Lin, S. (1998). Test of significance when data are curves. J. Amer. Statist. Assoc. 93 1007--1021.
Freund, Y. and Schapire, R. E. (1997). A decision-theoretic generalization of on-line learning and an application to boosting. J. Comput. System Sci. 55 119--139.
Garthwaite, P. H. and Buckland, S. T. (1992). Generating Monte Carlo confidence intervals by the Robbins--Monro process. Appl. Statist. 41 159--171.
Hall, P. (1985). Resampling a coverage pattern. Stochastic Process. Appl. 20 231--246.
Mathematical Reviews (MathSciNet):
MR808159
Hall, P. (1986). On the bootstrap and confidence intervals. Ann. Statist. 14 1431--1452.
Mathematical Reviews (MathSciNet):
MR868310
Hall, P. (1992). The Bootstrap and Edgeworth Expansion. Springer, New York.
Hall, P. and Presnell, B. (1999a). Intentionally biased bootstrap methods. J. R. Stat. Soc. Ser. B Stat. Methodol. 61 143--158.
Hall, P. and Presnell, B. (1999b). Biased bootstrap methods for reducing the effects of contamination. J. R. Stat. Soc. Ser. B Stat. Methodol. 61 661--680.
Hall, P. and Presnell, B. (1999c). Density estimation under constraints. J. Comput. Graph. Statist. 8 259--277.
Hall, P. and Wilson, S. R. (1991). Two guidelines for bootstrap hypothesis testing. Biometrics 47 757--762.
Härdle, W. (1989). Resampling for inference from curves. Bull. Inst. Internat. Statist. 53 53--64.
Härdle, W. (1990). Applied Nonparametric Regression. Cambridge Univ. Press.
Hesterberg, T. C. (1999). Bootstrap tilting confidence intervals and hypothesis tests. In Computer Science and Statistics: Proc. 31st Symposium on the Interface 389--393. Interface Foundation of North America, Inc., Fairfax Station, VA.
Hinkley, D. V. (1980). Likelihood as approximate pivotal distribution. Biometrika 67 287--292.
Mathematical Reviews (MathSciNet):
MR581726
Huang, H. (2002). Scenario generation for multivariate series data using the nearest neighbor bootstrap. Ph.D. dissertation, Dept. Decision Sciences and Engineering, Rensselaer Polytechnic Institute, Troy, New York.
Lahiri, S. N. (2003). On the impact of bootstrap in survey sampling and small-area estimation. Statist. Sci. 18 199--210 (this issue).
Lawless, J. (1982). Statistical Models and Methods for Lifetime Data. Wiley, New York.
Mathematical Reviews (MathSciNet):
MR640866
Lee, S. M. S. and Young, G. A. (2003). Prepivoting by weighted bootstrap iteration. Biometrika 90 393--410.
Lee, Y. D. and Lahiri, S. N. (2002). Least squares variogram fitting by spatial subsampling. J. R. Stat. Soc. Ser. B Stat. Methodol. 64 837--854.
Liu, R. Y. and Singh, K. (1997). Notions of limiting $P$ values based on data depth and bootstrap. J. Amer. Statist. Assoc. 92 266--277.
Mammen, E. (1993). Bootstrap and wild bootstrap for high-dimensional linear models. Ann. Statist. 21 255--285.
Martin, M. A. (1990). On bootstrap iteration for coverage correction in confidence intervals. J. Amer. Statist. Assoc. 85 1105--1118.
McCullagh, P. (2000). Resampling and exchangeable arrays. Bernoulli 6 285--301.
Newton, M. A. and Geyer, C. J. (1994). Bootstrap recycling: A Monte Carlo alternative to the nested bootstrap. J. Amer. Statist. Assoc. 89 905--912.
Owen, A. B. (1988). Empirical likelihood ratio confidence intervals for a single functional. Biometrika 75 237--249.
Mathematical Reviews (MathSciNet):
MR946049
Owen, A. B. (2001). Empirical Likelihood. Chapman and Hall/CRC, Boca Raton, FL.
Politis, D. N. (2003). The impact of bootstrap methods on time series analysis. Statist. Sci. 18 219--230 (this issue).
Politis, D. N., Paparoditis, E. and Romano, J. P. (1999). Resampling marked point processes. In Multivariate Analysis, Design of Experiments, and Survey Sampling (S. Ghosh, ed.) 163--185. Dekker, New York.
Politis, D. N., Romano, J. P. and Wolf, M. (1999). Subsampling. Springer, New York.
Putter, H. and van Zwet, W. R. (1996). Resampling: Consistency of substitution estimators. Ann. Statist. 24 2297--2318.
Rajagopalan, B. and Lall, U. (1999). A $k$-nearest-neighbor simulator for daily precipitation and other weather variables. Water Resources Res. 35 3089--3101.
Samworth, R. J. (2003). A note on methods of restoring consistency to the bootstrap. Biometrika. To appear.
Schapire, R. E., Freund, Y., Bartlett, P. and Lee, W. S. (1998). Boosting the margin: A new explanation for the effectiveness of voting methods. Ann. Statist. 26 1651--1686.
Severini, T. A. (2000). Likelihood Methods in Statistics. Clarendon, Oxford.
Shao, J. (2003). Impact of the bootstrap on sample surveys. Statist. Sci. 18 191--198 (this issue).
Shao, J. and Tu, D. (1995). The Jackknife and Bootstrap. Springer, New York.
Singh, K. (1981). On the asymptotic accuracy of Efron's bootstrap. Ann. Statist. 91 1187--1195.
Mathematical Reviews (MathSciNet):
MR630102
Stute, W., González Manteiga, W. and Presedo Quindimil, M. (1998). Bootstrap approximations in model checks for regression. J. Amer. Statist. Assoc. 93 141--149.
Ventura, V. (2002). Non-parametric bootstrap recycling. Statist. Comput. 12 261--273.
Wang, Y. D. and Wahba, G. (1995). Bootstrap confidence intervals for smoothing splines and their comparison to Bayesian confidence intervals. J. Statist. Comput. Simulation 51 263--279.
Wu, C.-F. J. (1986). Jackknife, bootstrap and other resampling methods in regression analysis (with discussion). Ann. Statist. 14 1261--1350.
Mathematical Reviews (MathSciNet):
MR868303