The Annals of Statistics

Some nonasymptotic results on resampling in high dimension, I: Confidence regions

Sylvain Arlot, Gilles Blanchard, and Etienne Roquain

Full-text: Open access

Enhanced PDF (304 KB)

Abstract

We study generalized bootstrap confidence regions for the mean of a random vector whose coordinates have an unknown dependency structure. The random vector is supposed to be either Gaussian or to have a symmetric and bounded distribution. The dimensionality of the vector can possibly be much larger than the number of observations and we focus on a nonasymptotic control of the confidence level, following ideas inspired by recent results in learning theory. We consider two approaches, the first based on a concentration principle (valid for a large class of resampling weights) and the second on a resampled quantile, specifically using Rademacher weights. Several intermediate results established in the approach based on concentration principles are of interest in their own right. We also discuss the question of accuracy when using Monte Carlo approximations of the resampled quantities.

Article information

Source
Ann. Statist., Volume 38, Number 1 (2010), 51-82.

Dates
First available in Project Euclid: 31 December 2009

Permanent link to this document
https://projecteuclid.org/euclid.aos/1262271609

Digital Object Identifier
doi:10.1214/08-AOS667

Mathematical Reviews number (MathSciNet)
MR2589316

Zentralblatt MATH identifier
1180.62066

Subjects
Primary: 62G15: Tolerance and confidence regions
Secondary: 62G09: Resampling methods

Keywords
Confidence regions high-dimensional data nonasymptotic error control resampling cross-validation concentration inequalities resampled quantile

Citation

Arlot, Sylvain; Blanchard, Gilles; Roquain, Etienne. Some nonasymptotic results on resampling in high dimension, I: Confidence regions. Ann. Statist. 38 (2010), no. 1, 51--82. doi:10.1214/08-AOS667. https://projecteuclid.org/euclid.aos/1262271609


Export citation

References

  • [1] Arlot, S. (2007). Resampling and Model Selection. Ph.D. thesis, Univ. Paris XI.
    Zentralblatt MATH: 1326.62097
    Digital Object Identifier: doi:10.1214/08-EJS196
  • [2] Arlot, S., Blanchard, G. and Roquain, É. (2010). Some nonasymptotic results on resampling in high dimension. II: Multiple tests. Ann. Statist. 38 83–99.
    Mathematical Reviews (MathSciNet): MR2589317
    Zentralblatt MATH: 1181.62055
    Digital Object Identifier: doi:10.1214/08-AOS668
    Project Euclid: euclid.aos/1262271610
  • [3] Baraud, Y. (2004). Confidence balls in Gaussian regression. Ann. Statist. 32 528–551.
    Mathematical Reviews (MathSciNet): MR2060168
    Zentralblatt MATH: 1093.62051
    Digital Object Identifier: doi:10.1214/009053604000000085
    Project Euclid: euclid.aos/1083178937
  • [4] Beran, R. (2003). The impact of the bootstrap on statistical algorithms and theory. Statist. Sci. 18 175–184.
    Mathematical Reviews (MathSciNet): MR2026078
    Digital Object Identifier: doi:10.1214/ss/1063994972
    Project Euclid: euclid.ss/1063994972
    Zentralblatt MATH: 1331.62177
  • [5] Beran, R. and Dümbgen, L. (1998). Modulation of estimators and confidence sets. Ann. Statist. 26 1826–1856.
    Zentralblatt MATH: 1073.62538
    Digital Object Identifier: doi:10.1214/aos/1024691359
    Project Euclid: euclid.aos/1024691359
  • [6] Cai, T. and Low, M. (2006). Adaptive confidence balls. Ann. Statist. 34 202–228.
    Mathematical Reviews (MathSciNet): MR2275240
    Zentralblatt MATH: 1091.62037
    Digital Object Identifier: doi:10.1214/009053606000000146
    Project Euclid: euclid.aos/1146576261
  • [7] Cirel’son, B. R., Ibragimov, I. A. and Sudakov, V. N. (1976). Norms of Gaussian sample functions. In Proceedings of the Third Japan–USSR Symposium on Probability Theory. Lecture Notes in Mathematics 550 20–41. Springer, Berlin.
    Mathematical Reviews (MathSciNet): MR458556
  • [8] Darvas, F., Rautiainen, M., Pantazis, D., Baillet, S., Benali, H., Mosher, J., Garnero, L. and Leahy, R. (2005). Investigations of dipole localization accuracy in MEG using the bootstrap. NeuroImage 25 355–368.
  • [9] DiCiccio, T. J. and Efron, B. (1996). Bootstrap confidence intervals. Statist. Sci. 11 189–228.
    Mathematical Reviews (MathSciNet): MR1436647
    Digital Object Identifier: doi:10.1214/ss/1032280214
    Project Euclid: euclid.ss/1032280214
    Zentralblatt MATH: 0955.62574
  • [10] Durot, C. and Rozenholc, Y. (2006). An adaptive test for zero mean. Math. Methods Statist. 15 26–60.
    Mathematical Reviews (MathSciNet): MR2225429
  • [11] Efron, B. (1979). Bootstrap methods: Another look at the jackknife. Ann. Statist. 7 1–26.
    Mathematical Reviews (MathSciNet): MR515681
    Zentralblatt MATH: 0406.62024
    Digital Object Identifier: doi:10.1214/aos/1176344552
    Project Euclid: euclid.aos/1176344552
  • [12] Fisher, R. A. (1935). The Design of Experiments. Oliver and Boyd, Edinburgh.
  • [13] Fromont, M. (2007). Model selection by bootstrap penalization for classification. Mach. Learn. 66 165–207.
  • [14] Ge, Y., Dudoit, S. and Speed, T. P. (2003). Resampling-based multiple testing for microarray data analysis. Test 12 1–77.
    Mathematical Reviews (MathSciNet): MR1993286
    Zentralblatt MATH: 1056.62117
    Digital Object Identifier: doi:10.1007/BF02595811
  • [15] Hall, P. (1992). The Bootstrap and Edgeworth Expansion. Springer, New York.
    Mathematical Reviews (MathSciNet): MR1145237
    Zentralblatt MATH: 0744.62026
  • [16] Hall, P. and Mammen, E. (1994). On general resampling algorithms and their performance in distribution estimation. Ann. Statist. 22 2011–2030.
    Mathematical Reviews (MathSciNet): MR1329180
    Zentralblatt MATH: 0828.62039
    Digital Object Identifier: doi:10.1214/aos/1176325769
    Project Euclid: euclid.aos/1176325769
  • [17] Hoffmann, M. and Lepski, O. (2002). Random rates in anisotropic regression. Ann. Statist. 30 325–396.
    Mathematical Reviews (MathSciNet): MR1902892
    Zentralblatt MATH: 1012.62042
    Digital Object Identifier: doi:10.1214/aos/1021379858
    Project Euclid: euclid.aos/1021379858
  • [18] Jerbi, K., Lachaux, J.-P., N’Diaye, K., Pantazis, D., Leahy, R. M., Garnero, L. and Baillet, S. (2007). Coherent neural representation of hand speed in humans revealed by MEG imaging. PNAS 104 7676–7681.
  • [19] Juditsky, A. and Lambert-Lacroix, S. (2003). Nonparametric confidence set estimation. Math. Methods Statist. 12 410–428.
    Mathematical Reviews (MathSciNet): MR2054156
  • [20] Koltchinskii, V. (2001). Rademacher penalties and structural risk minimization. IEEE Trans. Inform. Theory 47 1902–1914.
    Mathematical Reviews (MathSciNet): MR1842526
    Digital Object Identifier: doi:10.1109/18.930926
    Zentralblatt MATH: 1008.62614
  • [21] Lepski, O. V. (1999). How to improve the accuracy of estimation. Math. Methods Statist. 8 441–486.
    Mathematical Reviews (MathSciNet): MR1755896
    Zentralblatt MATH: 1033.62032
  • [22] Li, K.-C. (1989). Honest confidence regions for nonparametric regression. Ann. Statist. 17 1001–1008.
    Mathematical Reviews (MathSciNet): MR1015135
    Zentralblatt MATH: 0681.62047
    Digital Object Identifier: doi:10.1214/aos/1176347253
    Project Euclid: euclid.aos/1176347253
  • [23] Mason, D. M. and Newton, M. A. (1992). A rank statistics approach to the consistency of a general bootstrap. Ann. Statist. 20 1611–1624.
    Zentralblatt MATH: 0777.62045
    Digital Object Identifier: doi:10.1214/aos/1176348787
    Project Euclid: euclid.aos/1176348787
  • [24] Massart, P. (2007). Concentration Inequalities and Model Selection (Lecture Notes of the St-Flour Probability Summer School 2003). Lecture Notes in Mathematics 1896. Springer, Berlin.
    Mathematical Reviews (MathSciNet): MR2319879
    Zentralblatt MATH: 1170.60006
  • [25] McDiarmid, C. (1989). On the method of bounded differences. In Surveys in Combinatorics. London Mathematical Society Lecture Notes 141 148–188. Cambridge Univ. Press, Cambridge.
    Mathematical Reviews (MathSciNet): MR1036755
    Zentralblatt MATH: 0712.05012
  • [26] Pantazis, D., Nichols, T. E., Baillet, S. and Leahy, R. M. (2005). A comparison of random field theory and permutation methods for statistical analysis of MEG data. NeuroImage 25 383–394.
  • [27] Politis, D. N., Romano, J. P. and Wolf, M. (1999). Subsampling. Springer, New York.
    Mathematical Reviews (MathSciNet): MR1707286
  • [28] Præstgaard, J. and Wellner, J. A. (1993). Exchangeably weighted bootstraps of the general empirical process. Ann. Probab. 21 2053–2086.
    Zentralblatt MATH: 0792.62038
    Digital Object Identifier: doi:10.1214/aop/1176989011
    Project Euclid: euclid.aop/1176989011
  • [29] Robins, J. and van der Vaart, A. (2006). Adaptive nonparametric confidence sets. Ann. Statist. 34 229–253.
    Mathematical Reviews (MathSciNet): MR2275241
    Zentralblatt MATH: 1091.62039
    Digital Object Identifier: doi:10.1214/009053605000000877
    Project Euclid: euclid.aos/1146576262
  • [30] Romano, J. P. and Wolf, M. (2005). Exact and approximate stepdown methods for multiple hypothesis testing. J. Amer. Statist. Assoc. 100 94–108.
    Mathematical Reviews (MathSciNet): MR2156821
    Zentralblatt MATH: 1117.62416
    Digital Object Identifier: doi:10.1198/016214504000000539
  • [31] van der Vaart, A. W. and Wellner, J. A. (1996). Weak Convergence and Empirical Processes. Springer, New York.
    Mathematical Reviews (MathSciNet): MR1385671
    Zentralblatt MATH: 0862.60002
  • [32] Waberski, T., Gobbele, R., Kawohl, W., Cordes, C. and Buchner, H. (2003). Immediate cortical reorganization after local anesthetic block of the thumb: Source localization of somatosensory evoked potentials in human subjects. Neurosci. Lett. 347 151–154.

The Institute of Mathematical Statistics

The Annals of Statistics

New content alerts

Email RSS ToC RSS Article
  • You have access to this content.
  • You have partial access to this content.
  • You do not have access to this content.
Turn Off MathJax

What is MathJax?

More like this

+ See more