The Annals of Statistics

Confidence sets in sparse regression

Richard Nickl and Sara van de Geer

Full-text: Open access

Abstract

The problem of constructing confidence sets in the high-dimensional linear model with $n$ response variables and $p$ parameters, possibly $p\ge n$, is considered. Full honest adaptive inference is possible if the rate of sparse estimation does not exceed $n^{-1/4}$, otherwise sparse adaptive confidence sets exist only over strict subsets of the parameter spaces for which sparse estimators exist. Necessary and sufficient conditions for the existence of confidence sets that adapt to a fixed sparsity level of the parameter vector are given in terms of minimal $\ell^{2}$-separation conditions on the parameter space. The design conditions cover common coherence assumptions used in models for sparsity, including (possibly correlated) sub-Gaussian designs.

Article information

Source
Ann. Statist., Volume 41, Number 6 (2013), 2852-2876.

Dates
First available in Project Euclid: 17 December 2013

Permanent link to this document
https://projecteuclid.org/euclid.aos/1387313392

Digital Object Identifier
doi:10.1214/13-AOS1170

Mathematical Reviews number (MathSciNet)
MR3161450

Zentralblatt MATH identifier
1288.62108

Subjects
Primary: 62J05: Linear regression
Secondary: 62G15: Tolerance and confidence regions

Keywords
Composite testing problem high-dimensional inference detection boundary quadratic risk estimation

Citation

Nickl, Richard; van de Geer, Sara. Confidence sets in sparse regression. Ann. Statist. 41 (2013), no. 6, 2852--2876. doi:10.1214/13-AOS1170. https://projecteuclid.org/euclid.aos/1387313392


Export citation

References

  • Alexander, K. S. (1985). Rates of growth for weighted empirical processes. In Proceedings of the Berkeley Conference in Honor of Jerzy Neyman and Jack Kiefer, Vol. II (Berkeley, Calif., 1983). 2 475–493. Wadsworth, Belmont, CA.
  • Arias-Castro, E., Candès, E. J. and Plan, Y. (2011). Global testing under sparse alternatives: ANOVA, multiple comparisons and the higher criticism. Ann. Statist. 39 2533–2556.
  • Baraud, Y. (2004). Confidence balls in Gaussian regression. Ann. Statist. 32 528–551.
  • Beran, R. and Dümbgen, L. (1998). Modulation of estimators and confidence sets. Ann. Statist. 26 1826–1856.
  • Bickel, P. J., Ritov, Y. and Tsybakov, A. B. (2009). Simultaneous analysis of lasso and Dantzig selector. Ann. Statist. 37 1705–1732.
  • Boucheron, S., Lugosi, G. and Massart, P. (2013). Concentration Inequalities. A Nonasymptotic Theory of Independence. Oxford Univ. Press, London.
  • Bühlmann, P. and van de Geer, S. (2011). Statistics for High-Dimensional Data: Methods, Theory and Applications. Springer, Heidelberg.
  • Bull, A. D. and Nickl, R. (2013). Adaptive confidence sets in $L^{2}$. Probab. Theory Related Fields 156 889–919.
  • Cai, T. T. and Low, M. G. (2006). Adaptive confidence balls. Ann. Statist. 34 202–228.
  • Candès, E. and Tao, T. (2007). The Dantzig selector: Statistical estimation when $p$ is much larger than $n$. Ann. Statist. 35 2313–2351.
  • Dudley, R. M. (1967). The sizes of compact subsets of Hilbert space and continuity of Gaussian processes. J. Funct. Anal. 1 290–330.
  • Giné, E. and Nickl, R. (2009). An exponential inequality for the distribution function of the kernel density estimator, with applications to adaptive estimation. Probab. Theory Related Fields 143 569–596.
  • Giné, E. and Nickl, R. (2010). Confidence bands in density estimation. Ann. Statist. 38 1122–1170.
  • Hoffmann, M. and Lepski, O. (2002). Random rates in anisotropic regression. Ann. Statist. 30 325–396.
  • Hoffmann, M. and Nickl, R. (2011). On adaptive inference and confidence bands. Ann. Statist. 39 2383–2409.
  • Ingster, Y. I., Tsybakov, A. B. and Verzelen, N. (2010). Detection boundary in sparse regression. Electron. J. Stat. 4 1476–1526.
  • Javanmard, A. and Montanari, A. (2013). Confidence intervals and hypothesis testing for high-dimensional regression. Available at arXiv:1306.3171.
  • Juditsky, A. and Lambert-Lacroix, S. (2003). Nonparametric confidence set estimation. Math. Methods Statist. 12 410–428.
  • Li, K.-C. (1989). Honest confidence regions for nonparametric regression. Ann. Statist. 17 1001–1008.
  • Loh, P.-L. and Wainwright, M. J. (2012). High-dimensional regression with noisy and missing data: Provable guarantees with nonconvexity. Ann. Statist. 40 1637–1664.
  • Pötscher, B. M. (2009). Confidence sets based on sparse estimators are necessarily large. Sankhyā 71 1–18.
  • Pötscher, B. M. and Schneider, U. (2011). Distributional results for thresholding estimators in high-dimensional Gaussian regression models. Electron. J. Stat. 5 1876–1934.
  • Robins, J. and van der Vaart, A. (2006). Adaptive nonparametric confidence sets. Ann. Statist. 34 229–253.
  • Talagrand, M. (1996). New concentration inequalities in product spaces. Invent. Math. 126 505–563.
  • van de Geer, S. A. (2000). Empirical Processes in M-Estimation. Cambridge Univ. Press, Cambridge.
  • van de Geer, S. (2001). Least squares estimation with complexity penalties. Math. Methods Statist. 10 355–374.
  • van de Geer, S., Bühlmann, P. and Ritov, Y. (2013). On asymptotically optimal confidence regions and tests for high-dimensional models. Submitted. Available at arXiv:1303.0518.
  • van der Vaart, A. W. and Wellner, J. A. (1996). Weak Convergence and Empirical Processes. Springer Series in Statistics. Springer, New York.
  • Zhang, C. H. and Zhang, S. S. (2011). Confidence intervals for low-dimensional parameters with high-dimensional data. 2011. Available at arXiv:1110.2563v1.