Annals of Statistics
- Ann. Statist.
- Volume 43, Number 6 (2015), 2706-2737.
Estimation of functionals of sparse covariance matrices
Jianqing Fan, Philippe Rigollet, and Weichen Wang
Full-text: Open access
Abstract
High-dimensional statistical tests often ignore correlations to gain simplicity and stability leading to null distributions that depend on functionals of correlation matrices such as their Frobenius norm and other $\ell_{r}$ norms. Motivated by the computation of critical values of such tests, we investigate the difficulty of estimation the functionals of sparse correlation matrices. Specifically, we show that simple plug-in procedures based on thresholded estimators of correlation matrices are sparsity-adaptive and minimax optimal over a large class of correlation matrices. Akin to previous results on functional estimation, the minimax rates exhibit an elbow phenomenon. Our results are further illustrated in simulated data as well as an empirical study of data arising in financial econometrics.
Article information
Source
Ann. Statist., Volume 43, Number 6 (2015), 2706-2737.
Dates
Received: February 2015
Revised: June 2015
First available in Project Euclid: 7 October 2015
Permanent link to this document
https://projecteuclid.org/euclid.aos/1444222090
Digital Object Identifier
doi:10.1214/15-AOS1357
Mathematical Reviews number (MathSciNet)
MR3405609
Zentralblatt MATH identifier
1327.62338
Subjects
Primary: 62H12: Estimation
Secondary: 62H15: Hypothesis testing 62C20: Minimax procedures 62H25: Factor analysis and principal components; correspondence analysis
Keywords
Covariance matrix functional estimation high-dimensional testing minimax elbow effect
Citation
Fan, Jianqing; Rigollet, Philippe; Wang, Weichen. Estimation of functionals of sparse covariance matrices. Ann. Statist. 43 (2015), no. 6, 2706--2737. doi:10.1214/15-AOS1357. https://projecteuclid.org/euclid.aos/1444222090
References
- Amini, A. A. and Wainwright, M. J. (2009). High-dimensional analysis of semidefinite relaxations for sparse principal components. Ann. Statist. 37 2877–2921.Mathematical Reviews (MathSciNet): MR2541450
Zentralblatt MATH: 1173.62049
Digital Object Identifier: doi:10.1214/08-AOS664
Project Euclid: euclid.aos/1247836672 - Arias-Castro, E., Bubeck, S. and Lugosi, G. (2015). Detecting positive correlations in a multivariate sample. Bernoulli 21 209–241.Mathematical Reviews (MathSciNet): MR3322317
Digital Object Identifier: doi:10.3150/13-BEJ565
Project Euclid: euclid.bj/1426597068 - Bai, Z. and Saranadasa, H. (1996). Effect of high-dimension: By an example of a two sample problem. Statist. Sinica 6 311–329.
- Berthet, Q. and Rigollet, P. (2013a). Complexity theoretic lower bounds for sparse principal component detection. J. Mach. Learn. Res. 30 1046–1066.
- Berthet, Q. and Rigollet, P. (2013b). Optimal detection of sparse principal components in high-dimension. Ann. Statist. 41 1780–1815.Mathematical Reviews (MathSciNet): MR3127849
Zentralblatt MATH: 1277.62155
Digital Object Identifier: doi:10.1214/13-AOS1127
Project Euclid: euclid.aos/1378386239 - Bickel, P. J. and Levina, E. (2008a). Covariance regularization by thresholding. Ann. Statist. 36 2577–2604.Mathematical Reviews (MathSciNet): MR2485008
Zentralblatt MATH: 1196.62062
Digital Object Identifier: doi:10.1214/08-AOS600
Project Euclid: euclid.aos/1231165180 - Bickel, P. J. and Levina, E. (2008b). Regularized estimation of large covariance matrices. Ann. Statist. 36 199–227.Mathematical Reviews (MathSciNet): MR2387969
Zentralblatt MATH: 1132.62040
Digital Object Identifier: doi:10.1214/009053607000000758
Project Euclid: euclid.aos/1201877299 - Bickel, P. J. and Ritov, Y. (1988). Estimating integrated squared density derivatives: Sharp best order of convergence estimates. Sankhyā Ser. A 50 381–393.Mathematical Reviews (MathSciNet): MR1065550
- Birnbaum, A., Johnstone, I. M., Nadler, B. and Paul, D. (2013). Minimax bounds for sparse PCA with noisy high-dimensional data. Ann. Statist. 41 1055–1084.Mathematical Reviews (MathSciNet): MR3113803
Zentralblatt MATH: 1292.62071
Digital Object Identifier: doi:10.1214/12-AOS1014
Project Euclid: euclid.aos/1371150893 - Butucea, C. (2007). Goodness-of-fit testing and quadratic functional estimation from indirect observations. Ann. Statist. 35 1907–1930.Mathematical Reviews (MathSciNet): MR2363957
Zentralblatt MATH: 1126.62028
Digital Object Identifier: doi:10.1214/009053607000000118
Project Euclid: euclid.aos/1194461716 - Butucea, C. and Meziani, K. (2011). Quadratic functional estimation in inverse problems. Stat. Methodol. 8 31–41.Mathematical Reviews (MathSciNet): MR2741507
Zentralblatt MATH: 1213.62134
Digital Object Identifier: doi:10.1016/j.stamet.2010.05.002 - Cai, T. and Liu, W. (2011). Adaptive thresholding for sparse covariance matrix estimation. J. Amer. Statist. Assoc. 106 672–684.Mathematical Reviews (MathSciNet): MR2847949
Zentralblatt MATH: 1232.62086
Digital Object Identifier: doi:10.1198/jasa.2011.tm10560 - Cai, T. T. and Low, M. G. (2005). Nonquadratic estimators of a quadratic functional. Ann. Statist. 33 2930–2956.Mathematical Reviews (MathSciNet): MR2253108
Zentralblatt MATH: 1085.62055
Digital Object Identifier: doi:10.1214/009053605000000147
Project Euclid: euclid.aos/1140191679 - Cai, T. T. and Low, M. G. (2006). Optimal adaptive estimation of a quadratic functional. Ann. Statist. 34 2298–2325.Mathematical Reviews (MathSciNet): MR2291501
Zentralblatt MATH: 1110.62048
Digital Object Identifier: doi:10.1214/009053606000000849
Project Euclid: euclid.aos/1169571798 - Cai, T. T., Ma, Z. and Wu, Y. (2013). Sparse PCA: Optimal rates and adaptive estimation. Ann. Statist. 41 3074–3110.Mathematical Reviews (MathSciNet): MR3161458
Zentralblatt MATH: 1288.62099
Digital Object Identifier: doi:10.1214/13-AOS1178
Project Euclid: euclid.aos/1388545679 - Cai, T., Ma, Z. and Wu, Y. (2015). Optimal estimation and rank detection for sparse spiked covariance matrices. Probab. Theory Related Fields 161 781–815.Mathematical Reviews (MathSciNet): MR3334281
Digital Object Identifier: doi:10.1007/s00440-014-0562-z - Cai, T. T., Ren, Z. and Zhou, H. H. (2013). Optimal rates of convergence for estimating Toeplitz covariance matrices. Probab. Theory Related Fields 156 101–143.Mathematical Reviews (MathSciNet): MR3055254
Zentralblatt MATH: 06176807
Digital Object Identifier: doi:10.1007/s00440-012-0422-7 - Cai, T. T. and Yuan, M. (2012). Adaptive covariance matrix estimation through block thresholding. Ann. Statist. 40 2014–2042.Mathematical Reviews (MathSciNet): MR3059075
Zentralblatt MATH: 1257.62060
Digital Object Identifier: doi:10.1214/12-AOS999
Project Euclid: euclid.aos/1351602535 - Cai, T. T., Zhang, C.-H. and Zhou, H. H. (2010). Optimal rates of convergence for covariance matrix estimation. Ann. Statist. 38 2118–2144.Mathematical Reviews (MathSciNet): MR2676885
Digital Object Identifier: doi:10.1214/09-AOS752
Project Euclid: euclid.aos/1278861244 - Cai, T. T. and Zhou, H. H. (2012). Minimax estimation of large covariance matrices under $\ell_{1}$-norm. Statist. Sinica 22 1319–1349.Mathematical Reviews (MathSciNet): MR3027084
- Chen, S. X. and Qin, Y.-L. (2010). A two-sample test for high-dimensional data with applications to gene-set testing. Ann. Statist. 38 808–835.Mathematical Reviews (MathSciNet): MR2604697
Zentralblatt MATH: 1183.62095
Digital Object Identifier: doi:10.1214/09-AOS716
Project Euclid: euclid.aos/1266586615 - Donoho, D. L. and Nussbaum, M. (1990). Minimax quadratic estimation of a quadratic functional. J. Complexity 6 290–323.Mathematical Reviews (MathSciNet): MR1081043
Zentralblatt MATH: 0724.62039
Digital Object Identifier: doi:10.1016/0885-064X(90)90025-9 - Efromovich, S. and Low, M. (1996). On optimal adaptive estimation of a quadratic functional. Ann. Statist. 24 1106–1125.Mathematical Reviews (MathSciNet): MR1401840
Zentralblatt MATH: 0865.62024
Digital Object Identifier: doi:10.1214/aos/1032526959
Project Euclid: euclid.aos/1032526959 - El Karoui, N. (2008). Operator norm consistent estimation of large-dimensional sparse covariance matrices. Ann. Statist. 36 2717–2756.Mathematical Reviews (MathSciNet): MR2485011
Zentralblatt MATH: 1196.62064
Digital Object Identifier: doi:10.1214/07-AOS559
Project Euclid: euclid.aos/1231165183 - Fama, E. F. and French, K. R. (1993). Common risk factors in the returns on stocks and bonds. Journal of Financial Economics 33 3–56.
- Fan, J. (1991). On the estimation of quadratic functionals. Ann. Statist. 19 1273–1294.Mathematical Reviews (MathSciNet): MR1126325
Zentralblatt MATH: 0729.62076
Digital Object Identifier: doi:10.1214/aos/1176348249
Project Euclid: euclid.aos/1176348249 - Fan, J., Fan, Y. and Lv, J. (2008). High dimensional covariance matrix estimation using a factor model. J. Econometrics 147 186–197.Mathematical Reviews (MathSciNet): MR2472991
Digital Object Identifier: doi:10.1016/j.jeconom.2008.09.017 - Fan, J., Liao, Y. and Mincheva, M. (2011). High-dimensional covariance matrix estimation in approximate factor models. Ann. Statist. 39 3320–3356.Mathematical Reviews (MathSciNet): MR3012410
Zentralblatt MATH: 1246.62151
Digital Object Identifier: doi:10.1214/11-AOS944
Project Euclid: euclid.aos/1330958681 - Fan, J., Liao, Y. and Mincheva, M. (2013). Large covariance estimation by thresholding principal orthogonal complements. J. R. Stat. Soc. Ser. B. Stat. Methodol. 75 603–680.
- Fan, J., Rigollet, P. and Wang, W. (2015). Supplement to “Estimation of functionals of sparse covariance matrices.” DOI:10.1214/15-AOS1357SUPP.
- Foucart, S. and Rauhut, H. (2013). A Mathematical Introduction to Compressive Sensing. Birkhäuser/Springer, New York.
- Hall, P. and Marron, J. S. (1987). Estimation of integrated squared density derivatives. Statist. Probab. Lett. 6 109–115.Mathematical Reviews (MathSciNet): MR907270
- Ibragimov, I. A., Nemirovskiĭ, A. S. and Khas’minskiĭ, R. Z. (1987). Some problems of nonparametric estimation in Gaussian white noise. Theory Probab. Appl. 31 391–406.
- Johnstone, I. M. and Lu, A. Y. (2009). On consistency and sparsity for principal components analysis in high-dimensions. J. Amer. Statist. Assoc. 104 682–693.
- Jung, S. and Marron, J. S. (2009). PCA consistency in high-dimension, low sample size context. Ann. Statist. 37 4104–4130.Mathematical Reviews (MathSciNet): MR2572454
Zentralblatt MATH: 1191.62108
Digital Object Identifier: doi:10.1214/09-AOS709
Project Euclid: euclid.aos/1256303538 - Klemelä, J. (2006). Sharp adaptive estimation of quadratic functionals. Probab. Theory Related Fields 134 539–564.Mathematical Reviews (MathSciNet): MR2214904
Zentralblatt MATH: 1082.62032
Digital Object Identifier: doi:10.1007/s00440-005-0447-2 - Lam, C. and Fan, J. (2009). Sparsistency and rates of convergence in large covariance matrix estimation. Ann. Statist. 37 4254–4278.Mathematical Reviews (MathSciNet): MR2572459
Zentralblatt MATH: 1191.62101
Digital Object Identifier: doi:10.1214/09-AOS720
Project Euclid: euclid.aos/1256303543 - Levina, E. and Vershynin, R. (2012). Partial estimation of covariance matrices. Probab. Theory Related Fields 153 405–419.Mathematical Reviews (MathSciNet): MR2948681
Zentralblatt MATH: 06093305
Digital Object Identifier: doi:10.1007/s00440-011-0349-4 - Lintner, J. (1965). The valuation of risk assets and the selection of risky investments in stock portfolios and capital budgets. The Review of Economics and Statistics 47 13–37.
- Ma, Z. (2013). Sparse principal component analysis and iterative thresholding. Ann. Statist. 41 772–801.Mathematical Reviews (MathSciNet): MR3099121
Zentralblatt MATH: 1267.62074
Digital Object Identifier: doi:10.1214/13-AOS1097
Project Euclid: euclid.aos/1368018173 - Mossin, J. (1966). Equilibrium in a capital asset market. Econometrica 34 768–783.
- Nemirovski, A. (2000). Topics in nonparametric statistics. In Lectures on Probability Theory and Statistics (Saint-Flour, 1998). Lecture Notes in Math. 1738 85–277. Springer, Berlin.Mathematical Reviews (MathSciNet): MR1775640
Zentralblatt MATH: 0998.62033
Digital Object Identifier: doi:10.1007/BFb0106703 - Nemirovskiĭ, A. S. and Khas’minskiĭ, R. Z. (1987). Nonparametric estimation of the functionals of the products of a signal observed in white noise. Problemy Peredachi Informatsii 23 27–38.Mathematical Reviews (MathSciNet): MR914348
- Onatski, A., Moreira, M. J. and Hallin, M. (2013). Asymptotic power of sphericity tests for high-dimensional data. Ann. Statist. 41 1204–1231.Mathematical Reviews (MathSciNet): MR3113808
Zentralblatt MATH: 1293.62125
Digital Object Identifier: doi:10.1214/13-AOS1100
Project Euclid: euclid.aos/1371150898 - Paul, D. and Johnstone, I. M. (2012). Augmented sparse principal component analysis for high-dimensional data. Available at arXiv:1202.1242v1.arXiv: 1202.1242v1
- Pesaran, M. H. and Yamagata, T. (2012). Testing capm with a large number of assets. IZA Discussion Papers 6469, Institute for the Study of Labor.
- Ravikumar, P., Wainwright, M. J., Raskutti, G. and Yu, B. (2011). High-dimensional covariance estimation by minimizing $\ell_{1}$-penalized log-determinant divergence. Electron. J. Stat. 5 935–980.Mathematical Reviews (MathSciNet): MR2836766
Zentralblatt MATH: 1274.62190
Digital Object Identifier: doi:10.1214/11-EJS631
Project Euclid: euclid.ejs/1316092865 - Rigollet, P. and Tsybakov, A. B. (2012). Comment: “Minimax estimation of large covariance matrices under $\ell_{1}$-norm” [MR3027084]. Statist. Sinica 22 1358–1367.Mathematical Reviews (MathSciNet): MR3027087
- Rothman, A. J., Levina, E. and Zhu, J. (2009). Generalized thresholding of large covariance matrices. J. Amer. Statist. Assoc. 104 177–186.
- Sharpe, W. F. (1964). Capital asset prices: A theory of market equilibrium under conditions of risk. J. Finance 19 425–442.
- Srivastava, M. S. and Du, M. (2008). A test for the mean vector with fewer observations than the dimension. J. Multivariate Anal. 99 386–402.Mathematical Reviews (MathSciNet): MR2396970
Digital Object Identifier: doi:10.1016/j.jmva.2006.11.002 - Tsybakov, A. B. (2009). Introduction to Nonparametric Estimation. Springer, New York.Mathematical Reviews (MathSciNet): MR2724359
- Verzelen, N. (2012). Minimax risks for sparse regressions: Ultra-high-dimensional phenomenons. Electron. J. Stat. 6 38–90.Mathematical Reviews (MathSciNet): MR2879672
Zentralblatt MATH: 06166950
Digital Object Identifier: doi:10.1214/12-EJS666
Project Euclid: euclid.ejs/1327505822 - Vu, V. and Lei, J. (2012). Minimax rates of estimation for sparse PCA in high-dimensions. In Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics April 21–23, 2012, JMLR W&CP 22 1278–1286.
- Zou, H., Hastie, T. and Tibshirani, R. (2006). Sparse principal component analysis. J. Comput. Graph. Statist. 15 265–286.Mathematical Reviews (MathSciNet): MR2252527
Digital Object Identifier: doi:10.1198/106186006X113430
Supplemental materials
- Technical proofs Fan, Rigollet and Wang (2015) . This supplementary material contains the introduction to two-sample high-dimensional testing methods and the proofs of upper bounds that were omitted from the paper.Digital Object Identifier: doi:10.1214/15-AOS1357SUPP
- You have access to this content.
- You have partial access to this content.
- You do not have access to this content.
More like this
- Minimax estimation of linear and quadratic functionals on sparsity classes
Collier, Olivier, Comminges, Laëtitia, and Tsybakov, Alexandre B., Annals of Statistics, 2017 - Optimal rates of convergence for sparse covariance matrix estimation
Cai, T. Tony and Zhou, Harrison H., Annals of Statistics, 2012 - Sparsistency and rates of convergence in large covariance matrix estimation
Lam, Clifford and Fan, Jianqing, Annals of Statistics, 2009
- Minimax estimation of linear and quadratic functionals on sparsity classes
Collier, Olivier, Comminges, Laëtitia, and Tsybakov, Alexandre B., Annals of Statistics, 2017 - Optimal rates of convergence for sparse covariance matrix estimation
Cai, T. Tony and Zhou, Harrison H., Annals of Statistics, 2012 - Sparsistency and rates of convergence in large covariance matrix estimation
Lam, Clifford and Fan, Jianqing, Annals of Statistics, 2009 - Adapting to unknown sparsity by controlling the false discovery rate
Abramovich, Felix, Benjamini, Yoav, Donoho, David L., and Johnstone, Iain M., Annals of Statistics, 2006 - Minimax estimation of large precision matrices with bandable Cholesky factor
Liu, Yu and Ren, Zhao, Annals of Statistics, 2020 - Sparse permutation invariant covariance estimation
Rothman, Adam J., Bickel, Peter J., Levina, Elizaveta, and Zhu, Ji, Electronic Journal of Statistics, 2008 - Exact minimax estimation of the predictive density in sparse Gaussian models
Mukherjee, Gourab and Johnstone, Iain M., Annals of Statistics, 2015 - Large volatility matrix estimation with factor-based diffusion model for high-frequency financial data
Kim, Donggyu, Liu, Yi, and Wang, Yazhen, Bernoulli, 2018 - Estimating structured high-dimensional covariance and precision matrices: Optimal rates and adaptive estimation
Cai, T. Tony, Ren, Zhao, and Zhou, Harrison H., Electronic Journal of Statistics, 2016 - Optimal large-scale quantum state tomography with Pauli measurements
Cai, Tony, Kim, Donggyu, Wang, Yazhen, Yuan, Ming, and Zhou, Harrison H., Annals of Statistics, 2016