The Annals of Statistics
- Ann. Statist.
- Volume 41, Number 4 (2013), 1892-1921.
A simple bootstrap method for constructing nonparametric confidence bands for functions
Full-text: Open access
Abstract
Standard approaches to constructing nonparametric confidence bands for functions are frustrated by the impact of bias, which generally is not estimated consistently when using the bootstrap and conventionally smoothed function estimators. To overcome this problem it is common practice to either undersmooth, so as to reduce the impact of bias, or oversmooth, and thereby introduce an explicit or implicit bias estimator. However, these approaches, and others based on nonstandard smoothing methods, complicate the process of inference, for example, by requiring the choice of new, unconventional smoothing parameters and, in the case of undersmoothing, producing relatively wide bands. In this paper we suggest a new approach, which exploits to our advantage one of the difficulties that, in the past, has prevented an attractive solution to the problem—the fact that the standard bootstrap bias estimator suffers from relatively high-frequency stochastic error. The high frequency, together with a technique based on quantiles, can be exploited to dampen down the stochastic error term, leading to relatively narrow, simple-to-construct confidence bands.
Article information
Source
Ann. Statist. Volume 41, Number 4 (2013), 1892-1921.
Dates
First available in Project Euclid: 5 September 2013
Permanent link to this document
https://projecteuclid.org/euclid.aos/1378386242
Digital Object Identifier
doi:10.1214/13-AOS1137
Mathematical Reviews number (MathSciNet)
MR3127852
Zentralblatt MATH identifier
1277.62120
Subjects
Primary: 62G07: Density estimation 62G08: Nonparametric regression
Secondary: 62G09: Resampling methods
Keywords
Bandwidth bias bootstrap confidence interval conservative coverage coverage error kernel methods statistical smoothing
Citation
Hall, Peter; Horowitz, Joel. A simple bootstrap method for constructing nonparametric confidence bands for functions. Ann. Statist. 41 (2013), no. 4, 1892--1921. doi:10.1214/13-AOS1137. https://projecteuclid.org/euclid.aos/1378386242
References
- Beran, R. (1987). Prepivoting to reduce level error of confidence sets. Biometrika 74 457–468.
- Berry, S. M., Carroll, R. J. and Ruppert, D. (2002). Bayesian smoothing and regression splines for measurement error problems. J. Amer. Statist. Assoc. 97 160–169.Mathematical Reviews (MathSciNet): MR1947277
Digital Object Identifier: doi:10.1198/016214502753479301 - Bjerve, S., Doksum, K. A. and Yandell, B. S. (1985). Uniform confidence bounds for regression based on a simple moving average. Scand. J. Stat. 12 159–169.Mathematical Reviews (MathSciNet): MR808152
- Brown, L. D. and Levine, M. (2007). Variance estimation in nonparametric regression via the difference sequence method. Ann. Statist. 35 2219–2232.Mathematical Reviews (MathSciNet): MR2363969
Digital Object Identifier: doi:10.1214/009053607000000145
Project Euclid: euclid.aos/1194461728 - Buckley, M. J., Eagleson, G. K. and Silverman, B. W. (1988). The estimation of residual variance in nonparametric regression. Biometrika 75 189–199.
- Cai, T. T., Levine, M. and Wang, L. (2009). Variance function estimation in multivariate nonparametric regression with fixed design. J. Multivariate Anal. 100 126–136.Mathematical Reviews (MathSciNet): MR2460482
Digital Object Identifier: doi:10.1016/j.jmva.2008.03.007 - Cai, T. T. and Low, M. G. (2006). Adaptive confidence balls. Ann. Statist. 34 202–228.Mathematical Reviews (MathSciNet): MR2275240
Digital Object Identifier: doi:10.1214/009053606000000146
Project Euclid: euclid.aos/1146576261 - Chen, S. X. (1996). Empirical likelihood confidence intervals for nonparametric density estimation. Biometrika 83 329–341.
- Chen, S. X., Härdle, W. and Li, M. (2003). An empirical likelihood goodness-of-fit test for time series. J. R. Stat. Soc. Ser. B Stat. Methodol. 65 663–678.
- Claeskens, G. and Van Keilegom, I. (2003). Bootstrap confidence bands for regression curves and their derivatives. Ann. Statist. 31 1852–1884.Mathematical Reviews (MathSciNet): MR2036392
Digital Object Identifier: doi:10.1214/aos/1074290329
Project Euclid: euclid.aos/1074290329 - Dette, H., Munk, A. and Wagner, T. (1998). Estimating the variance in nonparametric regression—what is a reasonable choice? J. R. Stat. Soc. Ser. B Stat. Methodol. 60 751–764.
- Efron, B. and Tibshirani, R. J. (1993). An Introduction to the Bootstrap. Monographs on Statistics and Applied Probability 57. Chapman & Hall, New York.Mathematical Reviews (MathSciNet): MR1270903
- Eubank, R. L. and Speckman, P. L. (1993). Confidence bands in nonparametric regression. J. Amer. Statist. Assoc. 88 1287–1301.Mathematical Reviews (MathSciNet): MR1245362
Digital Object Identifier: doi:10.1080/01621459.1993.10476410 - Eubank, R. L. and Wang, S. (1994). Confidence regions in non-parametric regression. Scand. J. Stat. 21 147–158.Mathematical Reviews (MathSciNet): MR1294590
- Fan, J. and Yao, Q. (1998). Efficient estimation of conditional variance functions in stochastic regression. Biometrika 85 645–660.
- Gasser, T., Sroka, L. and Jennen-Steinmetz, C. (1986). Residual variance and residual pattern in nonlinear regression. Biometrika 73 625–633.
- Genovese, C. R. and Wasserman, L. (2005). Confidence sets for nonparametric wavelet regression. Ann. Statist. 33 698–729.Mathematical Reviews (MathSciNet): MR2163157
Digital Object Identifier: doi:10.1214/009053605000000011
Project Euclid: euclid.aos/1117114334 - Genovese, C. and Wasserman, L. (2008). Adaptive confidence bands. Ann. Statist. 36 875–905.Mathematical Reviews (MathSciNet): MR2396818
Digital Object Identifier: doi:10.1214/07-AOS500
Project Euclid: euclid.aos/1205420522 - Giné, E. and Nickl, R. (2010). Confidence bands in density estimation. Ann. Statist. 38 1122–1170.Mathematical Reviews (MathSciNet): MR2604707
Digital Object Identifier: doi:10.1214/09-AOS738
Project Euclid: euclid.aos/1266586625 - Hall, P. (1986). On the bootstrap and confidence intervals. Ann. Statist. 14 1431–1452.Mathematical Reviews (MathSciNet): MR868310
Digital Object Identifier: doi:10.1214/aos/1176350168
Project Euclid: euclid.aos/1176350168 - Hall, P. (1992a). Effect of bias estimation on coverage accuracy of bootstrap confidence intervals for a probability density. Ann. Statist. 20 675–694.Mathematical Reviews (MathSciNet): MR1165587
Digital Object Identifier: doi:10.1214/aos/1176348651
Project Euclid: euclid.aos/1176348651 - Hall, P. (1992b). On bootstrap confidence intervals in nonparametric regression. Ann. Statist. 20 695–711.Mathematical Reviews (MathSciNet): MR1165588
Digital Object Identifier: doi:10.1214/aos/1176348652
Project Euclid: euclid.aos/1176348652 - Hall, P. and Horowitz, J. (2013). Supplement to “A simple bootstrap method for constructing nonparametric confidence bands for functions.” DOI:10.1214/13-AOS1137SUPP.
- Hall, P., Kay, J. W. and Titterington, D. M. (1990). Asymptotically optimal difference-based estimation of variance in nonparametric regression. Biometrika 77 521–528.
- Hall, P. and Marron, J. S. (1990). On variance estimation in nonparametric regression. Biometrika 77 415–419.
- Hall, P. and Owen, A. B. (1993). Empirical likelihood confidence bands in density estimation. J. Comput. Graph. Statist. 2 273–289.
- Hall, P. and Titterington, D. M. (1988). On confidence bands in nonparametric density estimation and regression. J. Multivariate Anal. 27 228–254.Mathematical Reviews (MathSciNet): MR971184
Digital Object Identifier: doi:10.1016/0047-259X(88)90127-3 - Härdle, W. and Bowman, A. W. (1988). Bootstrapping in nonparametric regression: Local adaptive smoothing and confidence bands. J. Amer. Statist. Assoc. 83 102–110.Mathematical Reviews (MathSciNet): MR941002
- Härdle, W., Huet, S. and Jolivet, E. (1995). Better bootstrap confidence intervals for regression curve estimation. Statistics 26 287–306.Mathematical Reviews (MathSciNet): MR1365680
Digital Object Identifier: doi:10.1080/02331889508802498 - Härdle, W. and Marron, J. S. (1991). Bootstrap simultaneous error bars for nonparametric regression. Ann. Statist. 19 778–796.Mathematical Reviews (MathSciNet): MR1105844
Digital Object Identifier: doi:10.1214/aos/1176348120
Project Euclid: euclid.aos/1176348120 - Härdle, W., Huet, S., Mammen, E. and Sperlich, S. (2004). Bootstrap inference in semiparametric generalized additive models. Econometric Theory 20 265–300.Mathematical Reviews (MathSciNet): MR2044272
- Hoffmann, M. and Nickl, R. (2011). On adaptive inference and confidence bands. Ann. Statist. 39 2383–2409.Mathematical Reviews (MathSciNet): MR2906872
Digital Object Identifier: doi:10.1214/11-AOS903
Project Euclid: euclid.aos/1322663462 - Horowitz, J. L. and Spokoiny, V. G. (2001). An adaptive, rate-optimal test of a parametric mean-regression model against a nonparametric alternative. Econometrica 69 599–631.
- Komlós, J., Major, P. and Tusnády, G. (1976). An approximation of partial sums of independent RV’s, and the sample DF. II. Z. Wahrsch. Verw. Gebiete 34 33–58.
- Li, K.-C. (1989). Honest confidence regions for nonparametric regression. Ann. Statist. 17 1001–1008.Mathematical Reviews (MathSciNet): MR1015135
Digital Object Identifier: doi:10.1214/aos/1176347253
Project Euclid: euclid.aos/1176347253 - Loh, W.-Y. (1987). Calibrating confidence coefficients. J. Amer. Statist. Assoc. 82 155–162.Mathematical Reviews (MathSciNet): MR883343
Digital Object Identifier: doi:10.1080/01621459.1987.10478408 - Low, M. G. (1997). On nonparametric confidence intervals. Ann. Statist. 25 2547–2554.Mathematical Reviews (MathSciNet): MR1604412
Digital Object Identifier: doi:10.1214/aos/1030741084
Project Euclid: euclid.aos/1030741084 - Massart, P. (1989). Strong approximation for multivariate empirical and related processes, via KMT constructions. Ann. Probab. 17 266–291.Mathematical Reviews (MathSciNet): MR972785
Digital Object Identifier: doi:10.1214/aop/1176991508
Project Euclid: euclid.aop/1176991508 - McMurry, T. L. and Politis, D. N. (2008). Bootstrap confidence intervals in nonparametric regression with built-in bias correction. Statist. Probab. Lett. 78 2463–2469.Mathematical Reviews (MathSciNet): MR2462680
- Mendez, G. and Lohr, S. (2011). Estimating residual variance in random forest regression. Comput. Statist. Data Anal. 55 2937–2950.Mathematical Reviews (MathSciNet): MR2813057
- Müller, U. U., Schick, A. and Wefelmeyer, W. (2003). Estimating the error variance in nonparametric regression by a covariate-matched $U$-statistic. Statistics 37 179–188.Mathematical Reviews (MathSciNet): MR1986175
Digital Object Identifier: doi:10.1080/0233188031000078051 - Müller, H.-G. and Stadtmüller, U. (1987). Estimation of heteroscedasticity in regression analysis. Ann. Statist. 15 610–625.Mathematical Reviews (MathSciNet): MR888429
Digital Object Identifier: doi:10.1214/aos/1176350364
Project Euclid: euclid.aos/1176350364 - Müller, H.-G. and Stadtmüller, U. (1993). On variance function estimation with quadratic forms. J. Statist. Plann. Inference 35 213–231.Mathematical Reviews (MathSciNet): MR1220417
Digital Object Identifier: doi:10.1016/0378-3758(93)90046-9 - Müller, H.-G. and Zhao, P. L. (1995). On a semiparametric variance function model and a test for heteroscedasticity. Ann. Statist. 23 946–967.Mathematical Reviews (MathSciNet): MR1345208
Digital Object Identifier: doi:10.1214/aos/1176324630
Project Euclid: euclid.aos/1176324630 - Munk, A., Bissantz, N., Wagner, T. and Freitag, G. (2005). On difference-based variance estimation in nonparametric regression when the covariate is high dimensional. J. R. Stat. Soc. Ser. B Stat. Methodol. 67 19–41.Mathematical Reviews (MathSciNet): MR2136637
Digital Object Identifier: doi:10.1111/j.1467-9868.2005.00486.x - Neumann, M. H. (1994). Fully data-driven nonparametric variance estimators. Statistics 25 189–212.Mathematical Reviews (MathSciNet): MR1366825
Digital Object Identifier: doi:10.1080/02331889408802445 - Neumann, M. H. (1995). Automatic bandwidth choice and confidence intervals in nonparametric regression. Ann. Statist. 23 1937–1959.Mathematical Reviews (MathSciNet): MR1389859
Digital Object Identifier: doi:10.1214/aos/1034713641
Project Euclid: euclid.aos/1034713641 - Neumann, M. H. and Polzehl, J. (1998). Simultaneous bootstrap confidence bands in nonparametric regression. J. Nonparametr. Stat. 9 307–333.Mathematical Reviews (MathSciNet): MR1646905
Digital Object Identifier: doi:10.1080/10485259808832748 - Picard, D. and Tribouley, K. (2000). Adaptive confidence interval for pointwise curve estimation. Ann. Statist. 28 298–335.Mathematical Reviews (MathSciNet): MR1762913
Digital Object Identifier: doi:10.1214/aos/1016120374
Project Euclid: euclid.aos/1016120374 - Rice, J. (1984). Bandwidth choice for nonparametric regression. Ann. Statist. 12 1215–1230.Mathematical Reviews (MathSciNet): MR760684
Digital Object Identifier: doi:10.1214/aos/1176346788
Project Euclid: euclid.aos/1176346788 - Ruppert, D., Sheather, S. J. and Wand, M. P. (1995). An effective bandwidth selector for local least squares regression. J. Amer. Statist. Assoc. 90 1257–1270.Mathematical Reviews (MathSciNet): MR1379468
Digital Object Identifier: doi:10.1080/01621459.1995.10476630 - Ruppert, D. and Wand, M. P. (1994). Multivariate locally weighted least squares regression. Ann. Statist. 22 1346–1370.Mathematical Reviews (MathSciNet): MR1311979
Digital Object Identifier: doi:10.1214/aos/1176325632
Project Euclid: euclid.aos/1176325632 - Ruppert, D., Wand, M. P. and Carroll, R. J. (2003). Semiparametric Regression. Cambridge Series in Statistical and Probabilistic Mathematics 12. Cambridge Univ. Press, Cambridge.Mathematical Reviews (MathSciNet): MR1998720
- Schucany, W. R. and Sommers, J. P. (1977). Improvement of kernel type density estimators. J. Amer. Statist. Assoc. 72 420–423.Mathematical Reviews (MathSciNet): MR448691
Digital Object Identifier: doi:10.1080/01621459.1977.10481012 - Seifert, B., Gasser, T. and Wolf, A. (1993). Nonparametric estimation of residual variance revisited. Biometrika 80 373–383.
- Sun, J. and Loader, C. R. (1994). Simultaneous confidence bands for linear regression and smoothing. Ann. Statist. 22 1328–1345.Mathematical Reviews (MathSciNet): MR1311978
Digital Object Identifier: doi:10.1214/aos/1176325631
Project Euclid: euclid.aos/1176325631 - Tong, T. and Wang, Y. (2005). Estimating residual variance in nonparametric regression using least squares. Biometrika 92 821–830.
- Tusnády, G. (1977). A remark on the approximation of the sample $DF$ in the multidimensional case. Period. Math. Hungar. 8 53–55.
- Wang, Y. D. and Wahba, G. (1995). Bootstrap confidence-intervals and for smoothing splines and their comparison to Bayesian confidence-intervals. Comm. Statist. Simulation Comput. 51 263–279.
- Xia, Y. (1998). Bias-corrected confidence bands in nonparametric regression. J. R. Stat. Soc. Ser. B Stat. Methodol. 60 797–811.
Supplemental materials
- Supplementary material: Appendix B. The supplementary material in Appendix B.1 outlines theoretical properties underpinning our methodology, while Appendix B.2 contains a proof of Theorem 4.1.Digital Object Identifier: doi:10.1214/13-AOS1137SUPP
- You have access to this content.
- You have partial access to this content.
- You do not have access to this content.
More like this
- Effect of Bias Estimation on Coverage Accuracy of Bootstrap Confidence Intervals for a Probability Density
Hall, Peter, The Annals of Statistics, 1992 - Direct use of regression quantiles to construct confidence sets in linear models
Zhou, Kenneth Q. and Portnoy, Stephen L., The Annals of Statistics, 1996 - On wavelet methods for estimating smooth functions
Hall, Peter and Patil, Prakash, Bernoulli, 1995
- Effect of Bias Estimation on Coverage Accuracy of Bootstrap Confidence Intervals for a Probability Density
Hall, Peter, The Annals of Statistics, 1992 - Direct use of regression quantiles to construct confidence sets in linear models
Zhou, Kenneth Q. and Portnoy, Stephen L., The Annals of Statistics, 1996 - On wavelet methods for estimating smooth functions
Hall, Peter and Patil, Prakash, Bernoulli, 1995 - Bias correction and bootstrap methods for a spatial sampling scheme
Hall, Peter, Melville, Gavin, and Welsh, Alan H., Bernoulli, 2001 - Nonparametric likelihood ratio confidence bands for quantile functions from incomplete survival data
Li, Gang, Hollander, Myles, McKeague, Ian W., and Yang, Jie, The Annals of Statistics, 1996 - Normalizing Transformatins and Bootstrap Confidence Intervals
Konishi, Sadanori, The Annals of Statistics, 1991 - Inconsistency of bootstrap: The Grenander estimator
Sen, Bodhisattva, Banerjee, Moulinath, and Woodroofe, Michael, The Annals of Statistics, 2010 - Oracally efficient estimation of autoregressive error distribution with simultaneous confidence band
Wang, Jiangyan, Liu, Rong, Cheng, Fuxia, and Yang, Lijian, The Annals of Statistics, 2014 - Estimation and inference in generalized additive coefficient models for nonlinear interactions with high-dimensional covariates
Ma, Shujie, Carroll, Raymond J., Liang, Hua, and Xu, Shizhong, The Annals of Statistics, 2015 - Regression-type inference in nonparametric autoregression
Kreiss, Jens-Peter and Neumann, Michael H., The Annals of Statistics, 1998