Bernoulli

  • Bernoulli
  • Volume 19, Number 5B (2013), 2359-2388.

Optimal hypothesis testing for high dimensional covariance matrices

T. Tony Cai and Zongming Ma

Full-text: Open access

Abstract

This paper considers testing a covariance matrix $\Sigma$ in the high dimensional setting where the dimension $p$ can be comparable or much larger than the sample size $n$. The problem of testing the hypothesis $H_{0}:\Sigma=\Sigma_{0}$ for a given covariance matrix $\Sigma_{0}$ is studied from a minimax point of view. We first characterize the boundary that separates the testable region from the non-testable region by the Frobenius norm when the ratio between the dimension $p$ over the sample size $n$ is bounded. A test based on a $U$-statistic is introduced and is shown to be rate optimal over this asymptotic regime. Furthermore, it is shown that the power of this test uniformly dominates that of the corrected likelihood ratio test (CLRT) over the entire asymptotic regime under which the CLRT is applicable. The power of the $U$-statistic based test is also analyzed when $p/n$ is unbounded.

Article information

Source
Bernoulli, Volume 19, Number 5B (2013), 2359-2388.

Dates
First available in Project Euclid: 3 December 2013

Permanent link to this document
https://projecteuclid.org/euclid.bj/1386078606

Digital Object Identifier
doi:10.3150/12-BEJ455

Mathematical Reviews number (MathSciNet)
MR3160557

Zentralblatt MATH identifier
1281.62140

Keywords
correlation matrix covariance matrix high-dimensional data likelihood ratio test minimax hypothesis testing power testing covariance structure

Citation

Cai, T. Tony; Ma, Zongming. Optimal hypothesis testing for high dimensional covariance matrices. Bernoulli 19 (2013), no. 5B, 2359--2388. doi:10.3150/12-BEJ455. https://projecteuclid.org/euclid.bj/1386078606


Export citation

References

  • [1] Anderson, T.W. (2003). An Introduction to Multivariate Statistical Analysis, 3rd ed. Wiley Series in Probability and Statistics. Hoboken, NJ: Wiley-Interscience [John Wiley & Sons].
  • [2] Bai, Z., Jiang, D., Yao, J.F. and Zheng, S. (2009). Corrections to LRT on large-dimensional covariance matrix by RMT. Ann. Statist. 37 3822–3840.
  • [3] Birke, M. and Dette, H. (2005). A note on testing the covariance matrix for large dimension. Statist. Probab. Lett. 74 281–289.
  • [4] Cai, T.T. and Jiang, T. (2011). Limiting laws of coherence of random matrices with applications to testing covariance structure and construction of compressed sensing matrices. Ann. Statist. 39 1496–1525.
  • [5] Cai, T.T., Liu, W. and Xia, Y. (2011). Two-sample covariance matrix testing and support recovery. Technical report.
  • [6] Chen, S.X.and Li, J. (2012). Two sample tests for high dimensional covariance matrices. Ann. Statist. 40 908–940.
  • [7] Chen, S.X., Zhang, L.X. and Zhong, P.S. (2010). Tests for high-dimensional covariance matrices. J. Amer. Statist. Assoc. 105 810–819.
  • [8] Heyde, C.C. and Brown, B.M. (1970). On the departure from normality of a certain class of martingales. Ann. Math. Statist. 41 2161–2165.
  • [9] Hoeffding, W. (1963). Probability inequalities for sums of bounded random variables. J. Amer. Statist. Assoc. 58 13–30.
  • [10] Jiang, D., Jiang, T. and Yang, F. (2012). Likelihood ratio tests for covariance matrices of high-dimensional normal distributions. J. Statist. Plann. Inference 142 2241–2256.
  • [11] Johnstone, I.M. (2001). On the distribution of the largest eigenvalue in principal components analysis. Ann. Statist. 29 295–327.
  • [12] Ledoit, O. and Wolf, M. (2002). Some hypothesis tests for the covariance matrix when the dimension is large compared to the sample size. Ann. Statist. 30 1081–1102.
  • [13] Muirhead, R.J. (1982). Aspects of Multivariate Statistical Theory. Wiley Series in Probability and Mathematical Statistics. New York: Wiley.
  • [14] Nagao, H. (1973). On some test criteria for covariance matrix. Ann. Statist. 1 700–709.
  • [15] Onatski, A., Moreira, M.J. and Hallin, M. (2011). Asymptotic power of sphericity tests for high-dimensional data. Available at http://www.econ.cam.ac.uk/faculty/onatski/pubs/WPOnatskiMoreira.pdf.
  • [16] Roy, S.N. (1957). Some Aspects of Multivariate Analysis. New York: Wiley.
  • [17] Srivastava, M.S. (2005). Some tests concerning the covariance matrix in high dimensional data. J. Japan Statist. Soc. 35 251–272.
  • [18] Xiao, H. and Wu, W.B. (2011). Simultaneous inference on sample covariances. Available at arXiv:1109.0524v1.