Electronic Journal of Statistics

Test of independence for high-dimensional random vectors based on freeness in block correlation matrices

Zhigang Bao, Jiang Hu, Guangming Pan, and Wang Zhou

Full-text: Open access

Abstract

In this paper, we are concerned with the independence test for $k$ high-dimensional sub-vectors of a normal vector, with fixed positive integer $k$. A natural high-dimensional extension of the classical sample correlation matrix, namely block correlation matrix, is proposed for this purpose. We then construct the so-called Schott type statistic as our test statistic, which turns out to be a particular linear spectral statistic of the block correlation matrix. Interestingly, the limiting behavior of the Schott type statistic can be figured out with the aid of the Free Probability Theory and the Random Matrix Theory. Specifically, we will bring the so-called real second order freeness for Haar distributed orthogonal matrices, derived in Mingo and Popa (2013)[10], into the framework of this high-dimensional testing problem. Our test does not require the sample size to be larger than the total or any partial sum of the dimensions of the $k$ sub-vectors. Simulated results show the effect of the Schott type statistic, in contrast to those statistics proposed in Jiang and Yang (2013)[8] and Jiang, Bai and Zheng (2013)[7], is satisfactory. Real data analysis is also used to illustrate our method.

Article information

Source
Electron. J. Statist., Volume 11, Number 1 (2017), 1527-1548.

Dates
Received: September 2016
First available in Project Euclid: 19 April 2017

Permanent link to this document
https://projecteuclid.org/euclid.ejs/1492588988

Digital Object Identifier
doi:10.1214/17-EJS1259

Mathematical Reviews number (MathSciNet)
MR3635921

Zentralblatt MATH identifier
1362.62123

Subjects
Primary: 62H15: Hypothesis testing
Secondary: 46L54: Free probability and free operator algebras

Keywords
Block correlation matrix independence test high dimensional data Schott type statistic second order freeness haar distributed orthogonal matrices central limit theorem random matrices

Rights
Creative Commons Attribution 4.0 International License.

Citation

Bao, Zhigang; Hu, Jiang; Pan, Guangming; Zhou, Wang. Test of independence for high-dimensional random vectors based on freeness in block correlation matrices. Electron. J. Statist. 11 (2017), no. 1, 1527--1548. doi:10.1214/17-EJS1259. https://projecteuclid.org/euclid.ejs/1492588988


Export citation

References

  • [1] Anderson, T. (2003). An introduction to multivariate statistical analysis. Third Edition., Wiley New York.
  • [2] Bai, Z. D., Jiang, D. D., Yao, J. F. and Zheng, S. R. (2009). Corrections to LRT on large-dimensional covariance matrix by RMT., Ann. Statist., 37(6B), 3822–3840.
  • [3] Bai, Z. D. and Silverstein, J. W. (2004). CLT for linear spectral statistics of large-dimensional sample covariance matrices., Ann. Probab., 32(1A), 553–605.
  • [4] Cai, T. T. and Jiang, T. F. (2011). Limiting laws of coherence of random matrices with applications to testing covariance structure and construction of compressed sensing matrices., Ann. Statist., 39(3), 1496–1525.
  • [5] Collins, B., Mingo, J. A., Śniady, P. and Speicher, R. (2007). Second order freeness and fluctuations of Random Matrices: III. Higher order freeness and free cumulants, Doc. Math., 12, 1–70.
  • [6] Collins, B. and Śniady, P. (2006). Integration with respect to the Haar measure on unitary, orthogonal and symplectic group., Commun. Math. Phys., 264, 773–795.
  • [7] Jiang, D. D., Bai, Z. D. and Zheng, S. R. (2013). Testing the independence of sets of large-dimensional variables., Sci. China Math., 56(1), 135–147.
  • [8] Jiang, T. F. and Yang, F. (2013). Central limit theorems for classical likelihood ratio tests for high-dimensional normal distributions., Ann. Statist., 41(4), 2029–2074.
  • [9] Johansson, K. (1998). On fluctuations of eigenvalues of random Hermitian matrices., Duke Math. J., 91, 151–204.
  • [10] Mingo J. A., Popa M. (2013). Real second order freeness and Haar orthogonal matrices., J. Math. Phys., 54(5), 051701.
  • [11] Mingo, J. A., Speicher, R. (2006). Second order freeness and fluctuations of random matrices: I. Gaussian and Wishart matrices and cyclic Fock spaces., J. Funct. Anal., 235(1), 226–270.
  • [12] Mingo, J. A., Śniady, P. and Speicher, R. (2007). Second order freeness and fluctuations of random matrices: II. Unitary random matrices., Adv. Math., 209(1), 212–240.
  • [13] Muirhead, R. J. (1982). Aspects of Multivariate Statistical Theory., Wiley, New York.
  • [14] Johnstone, I. M. (2001). On the distribution of the largest eigenvalue in principal components analysis., Ann. Statist. 29, 295–327.
  • [15] Ledoit, O. and Wolf, M. (2002). Some hypothesis tests for the covariance matrix when the dimension is large compared to the sample size., Ann. Statist. 30, 1081–1102.
  • [16] Lytova, A. and Pastur, L. (2009). Central limit theorem for linear eigenvalue statistics of random matrices with independent entries., Ann. Probab., 37(5), 1778–1840.
  • [17] Pearson, Karl (1900). On the criterion that a given system of deviations from the probable in the case of a correlated system of variables is such that it can be reasonably supposed to have arisen from random sampling., Philosophical Magazine Series 5 50(302), 157–175.
  • [18] Rama, C. (2001). Empirical properties of asset returns: stylized facts and statistical issues., Quant. Finance, 1, 223–236.
  • [19] Rao, N. R., Mingo, J. A., Speicher, R. and Edelman, A. (2008). Statistical eigen-inference from large Wishart matrices., Ann. Statist., 2850–2885.
  • [20] Redelmeier, E. (2013). Real Second-Order Freeness and the Asymptotic Real Second-Order Freeness of Several Real Matrix Models., Int. Math. Res. Notices, 2014(12), 3353–3395.
  • [21] Schott, J. R. (2005). Testing for complete independence in high dimensions., Biometrika, 92, 951–956.
  • [22] Shcherbina, M. (2011). Central limit theorem for linear eigenvalue statistics of the Wigner and sample covariance random matrices., Math. Phys. Anal. Geom., 7(2), 176–192.
  • [23] Srivastava, M. S. (2005). Some Tests Concerning the Covariance Matrix in High Dimensional Data., J. Japan Statist. Soc., 35(2), 251–272.
  • [24] Srivastava, M. S. and Reid, N. (2012). Testing the structure of the covariance matrix with fewer observations than the dimension., J. Multivariate Anal., 112, 156–171.
  • [25] Sinai, Y. and Soshnikov, A. (1998). Central limit theorem for traces of large random symmetric matrices with independent matrix elements., Bol. Soc. Brasil. Mat., 29(1), 1–24.
  • [26] Voiculescu, D. (1991). Limit laws for random matrices and free products., Invent. Math., 104(1), 201–220.
  • [27] Wilks, S. S. (1935). On the independence of k sets of normally distributed statistical variables., Econometrica, 3, 309–326.
  • [28] Yang, Y. R, and Pan, G. M. (2015). Independence test for high dimensional data based on regularized canonical correlation coefficients., Ann. Statist., 43, 467–500.