The Annals of Statistics

Gaussian approximations and multiplier bootstrap for maxima of sums of high-dimensional random vectors

Victor Chernozhukov, Denis Chetverikov, and Kengo Kato

Full-text: Open access

Abstract

We derive a Gaussian approximation result for the maximum of a sum of high-dimensional random vectors. Specifically, we establish conditions under which the distribution of the maximum is approximated by that of the maximum of a sum of the Gaussian random vectors with the same covariance matrices as the original vectors. This result applies when the dimension of random vectors ($p$) is large compared to the sample size ($n$); in fact, $p$ can be much larger than $n$, without restricting correlations of the coordinates of these vectors. We also show that the distribution of the maximum of a sum of the random vectors with unknown covariance matrices can be consistently estimated by the distribution of the maximum of a sum of the conditional Gaussian random vectors obtained by multiplying the original vectors with i.i.d. Gaussian multipliers. This is the Gaussian multiplier (or wild) bootstrap procedure. Here too, $p$ can be large or even much larger than $n$. These distributional approximations, either Gaussian or conditional Gaussian, yield a high-quality approximation to the distribution of the original maximum, often with approximation error decreasing polynomially in the sample size, and hence are of interest in many applications. We demonstrate how our Gaussian approximations and the multiplier bootstrap can be used for modern high-dimensional estimation, multiple hypothesis testing, and adaptive specification testing. All these results contain nonasymptotic bounds on approximation errors.

Article information

Source
Ann. Statist., Volume 41, Number 6 (2013), 2786-2819.

Dates
First available in Project Euclid: 17 December 2013

Permanent link to this document
https://projecteuclid.org/euclid.aos/1387313390

Digital Object Identifier
doi:10.1214/13-AOS1161

Mathematical Reviews number (MathSciNet)
MR3161448

Zentralblatt MATH identifier
1292.62030

Subjects
Primary: 62E17: Approximations to distributions (nonasymptotic) 62F40: Bootstrap, jackknife and other resampling methods

Keywords
Dantzig selector Slepian Stein method maximum of vector sums high dimensionality anti-concentration

Citation

Chernozhukov, Victor; Chetverikov, Denis; Kato, Kengo. Gaussian approximations and multiplier bootstrap for maxima of sums of high-dimensional random vectors. Ann. Statist. 41 (2013), no. 6, 2786--2819. doi:10.1214/13-AOS1161. https://projecteuclid.org/euclid.aos/1387313390


Export citation

References

  • [1] Alquier, P. and Hebiri, M. (2011). Generalization of $\ell_{1}$ constraints for high dimensional regression problems. Statist. Probab. Lett. 81 1760–1765.
  • [2] Anderson, M. L. (2008). Multiple inference and gender differences in the effects of early intervention: A reevaluation of the Abecedarian, Perry Preschool, and early training projects. J. Amer. Statist. Assoc. 103 1481–1495.
  • [3] Arlot, S., Blanchard, G. and Roquain, E. (2010). Some nonasymptotic results on resampling in high dimension. I. Confidence regions. Ann. Statist. 38 51–82.
  • [4] Arlot, S., Blanchard, G. and Roquain, E. (2010). Some nonasymptotic results on resampling in high dimension. II. Multiple tests. Ann. Statist. 38 83–99.
  • [5] Belloni, A. and Chernozhukov, V. (2013). Least squares after model selection in high-dimensional sparse models. Bernoulli 19 521–547.
  • [6] Bickel, P. J., Ritov, Y. and Tsybakov, A. B. (2009). Simultaneous analysis of lasso and Dantzig selector. Ann. Statist. 37 1705–1732.
  • [7] Bretagnolle, J. and Massart, P. (1989). Hungarian constructions from the nonasymptotic viewpoint. Ann. Probab. 17 239–256.
  • [8] Bühlmann, P. and van de Geer, S. (2011). Statistics for High-Dimensional Data: Methods, Theory and Applications. Springer, Heidelberg.
  • [9] Candes, E. and Tao, T. (2007). The Dantzig selector: Statistical estimation when $p$ is much larger than $n$. Ann. Statist. 35 2313–2351.
  • [10] Chatterjee, S. (2005). A simple invariance theorem. Available at arXiv:math/0508213.
  • [11] Chatterjee, S. (2005). An error bound in the Sudakov–Fernique inequality. Available at arXiv:math/0510424.
  • [12] Chen, L., Goldstein, L. and Shao, Q. M. (2011). Normal Approximation by Stein’s Method. Springer, New York.
  • [13] Chernozhukov, V., Chetverikov, D. and Kato, K. (2012). Central limit theorems and multiplier bootstrap when $p$ is much larger than $n$. Available at arXiv:1212.6906.
  • [14] Chernozhukov, V., Chetverikov, D. and Kato, K. (2012). Gaussian approximation of suprema of empirical processes. Available at arXiv:1212.6906.
  • [15] Chernozhukov, V., Chetverikov, D. and Kato, K. (2012). Comparison and anti-concentration bounds for maxima of Gaussian random vectors. Available at arXiv:1301.4807.
  • [16] Chernozhukov, V., Chetverikov, D. and Kato, K. (2013). Supplement to “Gaussian approximations and multiplier bootstrap for maxima of sums of high-dimensional random vectors.” DOI:10.1214/13-AOS1161SUPP.
  • [17] Chetverikov, D. (2011). Adaptive test of conditional moment inequalities. Available at arXiv:1201.0167.
  • [18] Chetverikov, D. (2012). Testing regression monotonicity in econometric models. Available at arXiv:1212.6757.
  • [19] de la Peña, V. H., Lai, T. L. and Shao, Q.-M. (2009). Self-Normalized Processes: Limit Theory and Statistical Applications. Springer, Berlin.
  • [20] Dudley, R. M. (1999). Uniform Central Limit Theorems. Cambridge Studies in Advanced Mathematics 63. Cambridge Univ. Press, Cambridge.
  • [21] Dümbgen, L. and Spokoiny, V. G. (2001). Multiscale testing of qualitative hypotheses. Ann. Statist. 29 124–152.
  • [22] Fan, J., Hall, P. and Yao, Q. (2007). To how many simultaneous hypothesis tests can normal, Student’s $t$ or bootstrap calibration be applied? J. Amer. Statist. Assoc. 102 1282–1288.
  • [23] Frick, K., Marnitz, P. and Munk, A. (2012). Shape-constrained regularization by statistical multiresolution for inverse problems: Asymptotic analysis. Inverse Problems 28 065006, 31.
  • [24] Gautier, E. and Tsybakov, A. (2011). High-dimensional istrumental variables regression and confidence sets. Available at arXiv:1105.2454.
  • [25] Giné, E. and Nickl, R. (2010). Confidence bands in density estimation. Ann. Statist. 38 1122–1170.
  • [26] He, X. and Shao, Q.-M. (2000). On parameters of increasing dimensions. J. Multivariate Anal. 73 120–135.
  • [27] Juditsky, A. and Nemirovski, A. (2011). On verifiable sufficient conditions for sparse signal recovery via $\ell_{1}$ minimization. Math. Program. 127 57–88.
  • [28] Koltchinskii, V. (2009). The Dantzig selector and sparsity oracle inequalities. Bernoulli 15 799–828.
  • [29] Koltchinskii, V. I. (1994). Komlos–Major–Tusnady approximation for the general empirical process and Haar expansions of classes of functions. J. Theoret. Probab. 7 73–118.
  • [30] Komlós, J., Major, P. and Tusnády, G. (1975). An approximation of partial sums of independent $\mathrm{RV}$’s and the sample $\mathrm{DF}$. I. Z. Wahrsch. Verw. Gebiete 32 111–131.
  • [31] Leadbetter, M. R., Lindgren, G. and Rootzén, H. (1983). Extremes and Related Properties of Random Sequences and Processes. Springer, New York.
  • [32] Mammen, E. (1993). Bootstrap and wild bootstrap for high-dimensional linear models. Ann. Statist. 21 255–285.
  • [33] Panchenko, D. (2013). The Sherrington–Kirkpatrick Model. Springer, New York.
  • [34] Pollard, D. (2002). A User’s Guide to Measure Theoretic Probability. Cambridge Series in Statistical and Probabilistic Mathematics 8. Cambridge Univ. Press, Cambridge.
  • [35] Præstgaard, J. and Wellner, J. A. (1993). Exchangeably weighted bootstraps of the general empirical process. Ann. Probab. 21 2053–2086.
  • [36] Rio, E. (1994). Local invariance principles and their application to density estimation. Probab. Theory Related Fields 98 21–45.
  • [37] Röllin, A. (2011). Stein’s method in high dimensions with applications. Ann. Inst. Henri Poincaré Probab. Stat. 49 529–549.
  • [38] Romano, J. P. and Wolf, M. (2005). Exact and approximate stepdown methods for multiple hypothesis testing. J. Amer. Statist. Assoc. 100 94–108.
  • [39] Slepian, D. (1962). The one-sided barrier problem for Gaussian noise. Bell System Tech. J. 41 463–501.
  • [40] Stein, C. M. (1981). Estimation of the mean of a multivariate normal distribution. Ann. Statist. 9 1135–1151.
  • [41] Talagrand, M. (2003). Spin Glasses: A Challenge for Mathematicians: Cavity and Mean Field Models. Ergebnisse der Mathematik und Ihrer Grenzgebiete. 3. Folge. A Series of Modern Surveys in Mathematics [Results in Mathematics and Related Areas. 3rd Series. A Series of Modern Surveys in Mathematics] 46. Springer, Berlin.
  • [42] van der Vaart, A. W. and Wellner, J. A. (1996). Weak Convergence and Empirical Processes: With Applications to Statistics. Springer, New York.
  • [43] Ye, F. and Zhang, C.-H. (2010). Rate minimaxity of the Lasso and Dantzig selector for the $\ell_{q}$ loss in $\ell_{r}$ balls. J. Mach. Learn. Res. 11 3519–3540.

Supplemental materials

  • Supplementary material: Supplement to “Gaussian approximations and multiplier bootstrap for maxima of sums of high-dimensional random vectors”. This supplemental file contains the additional technical proofs, theoretical and simulation results.