Electronic Journal of Statistics

Confidence intervals for high-dimensional inverse covariance estimation

Jana Janková and Sara van de Geer

Full-text: Open access

Enhanced PDF (349 KB)

Abstract

We propose methodology for statistical inference for low-dimensional parameters of sparse precision matrices in a high-dimensional setting. Our method leads to a non-sparse estimator of the precision matrix whose entries have a Gaussian limiting distribution. Asymptotic properties of the novel estimator are analyzed for the case of sub-Gaussian observations under a sparsity assumption on the entries of the true precision matrix and regularity conditions. Thresholding the de-sparsified estimator gives guarantees for edge selection in the associated graphical model. Performance of the proposed method is illustrated in a simulation study.

Article information

Source
Electron. J. Statist., Volume 9, Number 1 (2015), 1205-1229.

Dates
Received: March 2014
First available in Project Euclid: 1 June 2015

Permanent link to this document
https://projecteuclid.org/euclid.ejs/1433195859

Digital Object Identifier
doi:10.1214/15-EJS1031

Mathematical Reviews number (MathSciNet)
MR3354336

Zentralblatt MATH identifier
1307.62015

Subjects
Primary: 62J07: Ridge regression; shrinkage estimators
Secondary: 62F12: Asymptotic properties of estimators

Keywords
Confidence intervals graphical Lasso high-dimensional precision matrix sparsity

Citation

Janková, Jana; van de Geer, Sara. Confidence intervals for high-dimensional inverse covariance estimation. Electron. J. Statist. 9 (2015), no. 1, 1205--1229. doi:10.1214/15-EJS1031. https://projecteuclid.org/euclid.ejs/1433195859


Export citation

References

  • [1] Belloni, A., Chernozhukov, V. and Wang, L. (2011). Square-root Lasso: Pivotal recovery of sparse signals via conic programming., Biometrika 98 791–806.
    Mathematical Reviews (MathSciNet): MR2860324
    Zentralblatt MATH: 1228.62083
    Digital Object Identifier: doi:10.1093/biomet/asr043
  • [2] Belloni, A., Chernozhukov, V. and Hansen, C. (2014). Inference on treatment effects after selection amongst high-dimensional controls., Review of Economic Studies 81 608–650.
    Mathematical Reviews (MathSciNet): MR3207983
    Zentralblatt MATH: 1153.68388
    Digital Object Identifier: doi:10.1093/restud/rdt044
  • [3] Berk, R., Brown, L., Buja, A., Zhang, K. and Zhao, L. (2013). Valid post-selection inference., Annals of Statistics 41 802–837.
    Mathematical Reviews (MathSciNet): MR3099122
    Zentralblatt MATH: 1267.62080
    Digital Object Identifier: doi:10.1214/12-AOS1077
    Project Euclid: euclid.aos/1369836961
  • [4] Bickel, P. J. and Levina, E. (2008). Covariance regularization by thresholding., Annals of Statistics 36 2577–2604.
    Mathematical Reviews (MathSciNet): MR2485008
    Zentralblatt MATH: 1196.62062
    Digital Object Identifier: doi:10.1214/08-AOS600
    Project Euclid: euclid.aos/1231165180
  • [5] Bühlmann, P. andvan de Geer, S. (2011)., Statistics for High-Dimensional Data. Springer.
    Mathematical Reviews (MathSciNet): MR2807761
  • [6] Cai, T., Liu, W. and Luo, X. (2011). A constrained l1 minimization approach to sparse precision matrix estimation., Journal of the American Statistical Association 106 594–607.
    Mathematical Reviews (MathSciNet): MR2847973
    Zentralblatt MATH: 1232.62087
    Digital Object Identifier: doi:10.1198/jasa.2011.tm10155
  • [7] Candes, E. and Tao, T. (2007). The Dantzig selector: Statistical estimation when p is much larger than n., Annals of Statistics 35 2313–2351.
    Mathematical Reviews (MathSciNet): MR2382644
    Zentralblatt MATH: 1139.62019
    Digital Object Identifier: doi:10.1214/009053606000001523
    Project Euclid: euclid.aos/1201012958
  • [8] Chatterjee, A. and Lahiri, S. N. (2011). Bootstrapping Lasso estimators., Journal of the American Statistical Association 106 608–625.
    Mathematical Reviews (MathSciNet): MR2847974
    Zentralblatt MATH: 1232.62088
    Digital Object Identifier: doi:10.1198/jasa.2011.tm10159
  • [9] Chatterjee, A. and Lahiri, S. N. (2013). Rates of convergence of the adaptive Lasso estimators to the oracle distribution and higher order refinements by the bootstrap., Annals of Statistics 41.
    Mathematical Reviews (MathSciNet): MR3113809
    Zentralblatt MATH: 1293.62153
    Digital Object Identifier: doi:10.1214/13-AOS1106
    Project Euclid: euclid.aos/1371150899
  • [10] d’Aspremont, A., Banerjee, O. and El Ghaoui, L. (2008). First-order methods for sparse covariance selection., SIAM J. Matrix Anal. Appl. 30 56–66.
    Mathematical Reviews (MathSciNet): MR2399568
    Digital Object Identifier: doi:10.1137/060670985
  • [11] Friedman, J., Hastie, T. and Tibshirani, R. (2008). Sparse inverse covariance estimation with the graphical lasso., Biostatistics 9 432– 441.
  • [12] Greene, W. H. (2011)., Econometric Analysis. Prentice Hall.
  • [13] Javanmard, A. and Montanari, A. (2013a). Confidence intervals and hypothesis testing for high-dimensional regression., ArXiv:1306.3171.
  • [14] Javanmard, A. and Montanari, A. (2013b). Model selection for high-dimensional regression under the generalized irrepresentability condition. In, Advances in Neural Information Processing Systems 26 (C. j. c. Burges, L. Bottou, M. Welling, Z. Ghahramani and K. q. Weinberger, eds.) 3012–3020.
  • [15] Johnstone, I. M. (2001). On the distribution of the largest eigenvalue in principal components analysis., Annals of Statistics 29 295–327.
    Mathematical Reviews (MathSciNet): MR1863961
    Zentralblatt MATH: 1016.62078
    Digital Object Identifier: doi:10.1214/aos/1009210544
    Project Euclid: euclid.aos/1009210544
  • [16] Knight, K. and Fu, W. (2000). Asymptotics for lasso-type estimators., Annals of Statistics 28 1356–1378.
    Mathematical Reviews (MathSciNet): MR1805787
    Zentralblatt MATH: 1105.62357
    Digital Object Identifier: doi:10.1214/aos/1015957397
    Project Euclid: euclid.aos/1015957397
  • [17] Lauritzen, S. L. (1996)., Graphical Models. Clarendon Press, Oxford.
    Mathematical Reviews (MathSciNet): MR1419991
  • [18] Leeb, H. and Pötscher, B. M. (2005). Model selection and inference: Facts and fiction., Econometric Theory 21 21–59.
    Mathematical Reviews (MathSciNet): MR2153856
    Zentralblatt MATH: 1085.62004
    Digital Object Identifier: doi:10.1017/S0266466605050036
  • [19] Leeb, H. and Pötscher, B. M. (2006). Can one estimate the conditional distribution of post-model-selection estimators?, Annals of Statistics 34 2554–2591.
    Mathematical Reviews (MathSciNet): MR2291510
    Digital Object Identifier: doi:10.1214/009053606000000821
    Project Euclid: euclid.aos/1169571807
  • [20] Meinshausen, N. (2013). Assumption-free confidence intervals for groups of variables in sparse high-dimensional regression., ArXiv:1309.3489.
  • [21] Meinshausen, N. and Bühlmann, P. (2006). High-dimensional graphs and variable selection with the Lasso., Annals of Statistics 34 1436–1462.
    Mathematical Reviews (MathSciNet): MR2278363
    Zentralblatt MATH: 1113.62082
    Digital Object Identifier: doi:10.1214/009053606000000281
    Project Euclid: euclid.aos/1152540754
  • [22] Meinshausen, N., Meier, L. and Bühlmann, P. (2009). P-values for high-dimensional regression., Journal of the American Statistical Association 104 1671–1681.
    Mathematical Reviews (MathSciNet): MR2750584
    Zentralblatt MATH: 1205.62089
    Digital Object Identifier: doi:10.1198/jasa.2009.tm08647
  • [23] Negahban, S. N., Ravikumar, P., Wainwright, M. J. and Yu, B. (2010). A unified framework for high-dimensional analysis of M-estimators with decomposable regularizers., Statistical Science 27 538–557.
    Mathematical Reviews (MathSciNet): MR3025133
    Digital Object Identifier: doi:10.1214/12-STS400
    Project Euclid: euclid.ss/1356098555
  • [24] Ng, B., G. Varoquaux, J.-B. P. andThirion, B. (2013). A novel sparse group gaussian graphical model for functional connectivity estimation., Information Processing in Medical Imaging.
  • [25] Nickl, R. andvan de Geer, S. (2012). Confidence sets in sparse regression., Annals of Statistics 41 2852–2876.
    Mathematical Reviews (MathSciNet): MR3161450
    Zentralblatt MATH: 1288.62108
    Digital Object Identifier: doi:10.1214/13-AOS1170
    Project Euclid: euclid.aos/1387313392
  • [26] Ravikumar, P., Raskutti, G., Wainwright, M. J. and Yu, B. (2008). High-dimensional covariance estimation by minimizing l1-penalized log-determinant divergence., Electronic Journal of Statistics 5 935–980.
    Mathematical Reviews (MathSciNet): MR2836766
    Zentralblatt MATH: 1274.62190
    Digital Object Identifier: doi:10.1214/11-EJS631
    Project Euclid: euclid.ejs/1316092865
  • [27] Ren, Z., Sun, T., Zhang, C. H. and Zhou, H. H. (2013). Asymptotic normality and optimalities in estimation of large Gaussian graphical model., ArXiv:1309.6024.
  • [28] Rothman, A. J., Bickel, P. J., Levina, E. and Zhu, J. (2008). Sparse permutation invariant covariance estimation., Electronic Journal of Statistics 2 494–515.
    Mathematical Reviews (MathSciNet): MR2417391
    Zentralblatt MATH: 06165702
    Digital Object Identifier: doi:10.1214/08-EJS176
    Project Euclid: euclid.ejs/1214491853
  • [29] Schäfer, J. and Strimmer, K. (2005). A shrinkage approach to large-scale covariance matrix estimation and implications for functional genomics., Statistical Applications in Genetics and Molecular Biology 4.
    Mathematical Reviews (MathSciNet): MR2183942
  • [30] Städler, N. and Mukherjee, S. (2013). Two-sample testing in high-dimensional models., Annals of Applied Statistics 7 1837–2457.
  • [31] Sun, T. and Zhang, C. H. (2012). Sparse matrix inversion with scaled Lasso., The Journal of Machine Learning Research 14 3385–3418.
    Mathematical Reviews (MathSciNet): MR3144466
    Zentralblatt MATH: 06378058
  • [32] van de Geer, S. (2014a). Statistical theory for high-dimensional models., ArXiv:1309.3489.
  • [33] van de Geer, S. (2014b). Worst possible sub-directions in high-dimensional models., ArXiv:1403.7023.
    Mathematical Reviews (MathSciNet): MR1382516
    Digital Object Identifier: doi:10.1051/ps:1997101
  • [34] van de Geer, S. A. and Bühlmann, P. (2009). On the conditions used to prove oracle results for the Lasso., Electronic Journal of Statistics 3 1360–1392.
    Mathematical Reviews (MathSciNet): MR2576316
    Zentralblatt MATH: 1106.62029
    Digital Object Identifier: doi:10.1214/09-EJS506
    Project Euclid: euclid.ejs/1260801227
  • [35] van de Geer, S., Bühlmann, P., Ritov, Y. and Dezeure, R. (2013). On asymptotically optimal confidence regions and tests for high-dimensional models., Annals of Statistics 42 1166–1202.
    Mathematical Reviews (MathSciNet): MR3224285
    Zentralblatt MATH: 1305.62259
    Digital Object Identifier: doi:10.1214/14-AOS1221
    Project Euclid: euclid.aos/1403276911
  • [36] Wasserman, L. and Roeder, K. (2009). High dimensional variable selection., Annals of Statistics 37 2178.
    Mathematical Reviews (MathSciNet): MR2543689
    Zentralblatt MATH: 1173.62054
    Digital Object Identifier: doi:10.1214/08-AOS646
    Project Euclid: euclid.aos/1247663752
  • [37] Yuan, M. (2010). High dimensional inverse covariance matrix estimation via linear programming., The Journal of Machine Learning Research 11 2261–2286.
    Mathematical Reviews (MathSciNet): MR2719856
    Zentralblatt MATH: 1242.62043
  • [38] Yuan, M. and Lin, Y. (2007). Model selection and estimation in the Gaussian graphical model., Biometrika 1–17.
    Mathematical Reviews (MathSciNet): MR2367824
    Zentralblatt MATH: 1142.62408
    Digital Object Identifier: doi:10.1093/biomet/asm018
  • [39] Zhang, C. H. and Zhang, S. S. (2014). Confidence intervals for low-dimensional parameters in high-dimensional linear models., Journal of the Royal Statistical Society: Series B 76 217–242.
    Mathematical Reviews (MathSciNet): MR3153940
    Digital Object Identifier: doi:10.1111/rssb.12026
  • [40] Zhao, P. and Yu, B. (2006). On model selection consistency of Lasso., Journal of Machine Learning Research 7 2541–2563.
    Mathematical Reviews (MathSciNet): MR2274449
    Zentralblatt MATH: 1222.62008

The Institute of Mathematical Statistics and the Bernoulli Society

Electronic Journal of Statistics

New content alerts

Email RSS ToC RSS Article
  • You have access to this content.
  • You have partial access to this content.
  • You do not have access to this content.
Turn Off MathJax

What is MathJax?

More like this

+ See more