The Annals of Statistics

Spectral statistics of large dimensional Spearman’s rank correlation matrix and its application

Zhigang Bao, Liang-Ching Lin, Guangming Pan, and Wang Zhou

Full-text: Open access

Abstract

Let $\mathbf{Q}=(Q_{1},\ldots,Q_{n})$ be a random vector drawn from the uniform distribution on the set of all $n!$ permutations of $\{1,2,\ldots,n\}$. Let $\mathbf{Z}=(Z_{1},\ldots,Z_{n})$, where $Z_{j}$ is the mean zero variance one random variable obtained by centralizing and normalizing $Q_{j}$, $j=1,\ldots,n$. Assume that $\mathbf{X}_{i},i=1,\ldots,p$ are i.i.d. copies of $\frac{1}{\sqrt{p}}\mathbf{Z}$ and $X=X_{p,n}$ is the $p\times n$ random matrix with $\mathbf{X}_{i}$ as its $i$th row. Then $S_{n}=XX^{*}$ is called the $p\times n$ Spearman’s rank correlation matrix which can be regarded as a high dimensional extension of the classical nonparametric statistic Spearman’s rank correlation coefficient between two independent random variables. In this paper, we establish a CLT for the linear spectral statistics of this nonparametric random matrix model in the scenario of high dimension, namely, $p=p(n)$ and $p/n\to c\in(0,\infty)$ as $n\to\infty$. We propose a novel evaluation scheme to estimate the core quantity in Anderson and Zeitouni’s cumulant method in [Ann. Statist. 36 (2008) 2553–2576] to bypass the so-called joint cumulant summability. In addition, we raise a two-step comparison approach to obtain the explicit formulae for the mean and covariance functions in the CLT. Relying on this CLT, we then construct a distribution-free statistic to test complete independence for components of random vectors. Owing to the nonparametric property, we can use this test on generally distributed random variables including the heavy-tailed ones.

Article information

Source
Ann. Statist., Volume 43, Number 6 (2015), 2588-2623.

Dates
Received: October 2014
Revised: June 2015
First available in Project Euclid: 7 October 2015

Permanent link to this document
https://projecteuclid.org/euclid.aos/1444222086

Digital Object Identifier
doi:10.1214/15-AOS1353

Mathematical Reviews number (MathSciNet)
MR3405605

Zentralblatt MATH identifier
1328.15046

Subjects
Primary: 15B52: Random matrices
Secondary: 62H10: Distribution of statistics

Keywords
Spearman’s rank correlation matrix nonparametric method linear spectral statistics central limit theorem independence test

Citation

Bao, Zhigang; Lin, Liang-Ching; Pan, Guangming; Zhou, Wang. Spectral statistics of large dimensional Spearman’s rank correlation matrix and its application. Ann. Statist. 43 (2015), no. 6, 2588--2623. doi:10.1214/15-AOS1353. https://projecteuclid.org/euclid.aos/1444222086


Export citation

References

  • [1] Anderson, G. W. and Zeitouni, O. (2008). A CLT for regularized sample covariance matrices. Ann. Statist. 36 2553–2576.
  • [2] Bai, Z. and Zhou, W. (2008). Large sample covariance matrices without independence structures in columns. Statist. Sinica 18 425–442.
  • [3] Bai, Z. D. and Silverstein, J. W. (2006). Spectral Analysis of Large Dimensional Random Matrices, 2nd ed. Science Press, Beijing.
  • [4] Bao, Z., Pan, G. and Zhou, W. (2012). Tracy–Widom law for the extreme eigenvalues of sample correlation matrices. Electron. J. Probab. 17 1–32.
  • [5] Bao, Z. G., Lin, L. C., Pan, G. M. and Zhou, W. (2015). Supplement to “Spectral statistics of large dimensional Spearman’s rank correlation matrix and its application.” DOI:10.1214/15-AOS1353SUPP.
  • [6] Bose, A. and Sen, A. (2008). Another look at the moment method for large dimensional random matrices. Electron. J. Probab. 13 588–628.
  • [7] Brillinger, D. R. (2001). Time Series. Classics in Applied Mathematics 36. SIAM, Philadelphia, PA.
  • [8] Costin, O. and Lebowitz, J. L. (1995). Gaussian fluctuation in random matrices. Phys. Rev. Lett. 75 69–72.
  • [9] Jiang, T. (2004). The asymptotic distributions of the largest entries of sample correlation matrices. Ann. Appl. Probab. 14 865–880.
  • [10] Jiang, T. and Yang, F. (2013). Central limit theorems for classical likelihood ratio tests for high-dimensional normal distributions. Ann. Statist. 41 2029–2074.
  • [11] Ledoit, O. and Wolf, M. (2002). Some hypothesis tests for the covariance matrix when the dimension is large compared to the sample size. Ann. Statist. 30 1081–1102.
  • [12] Pan, G. M. and Zhou, W. (2008). Central limit theorem for signal-to-interference ratio of reduced rank linear receiver. Ann. Appl. Probab. 18 1232–1270.
  • [13] Pillai, N. S. and Yin, J. (2012). Edge universality of correlation matrices. Ann. Statist. 40 1737–1763.
  • [14] Rosenblatt, M. (1985). Stationary Sequences and Random Fields. Birkhäuser, Boston, MA.
  • [15] Schott, J. R. (2005). Testing for complete independence in high dimensions. Biometrika 92 951–956.
  • [16] Sinai, Y. and Soshnikov, A. (1998). Central limit theorem for traces of large random symmetric matrices with independent matrix elements. Bol. Soc. Bras. Mat. 29 1–24.
  • [17] Soshnikov, A. (2002). Gaussian limit for determinantal random point fields. Ann. Probab. 30 171–187.
  • [18] Wigner, E. P. (1955). Characteristic vectors of bordered matrices with infinite dimensions. Ann. of Math. (2) 62 548–564.
  • [19] Wu, W. B. and Shao, X. (2004). Limit theorems for iterated random functions. J. Appl. Probab. 41 425–436.
  • [20] Xiao, H. and Wu, W. B. (2011). Asymptotic inference of autocovariances of stationary processes. Available at arXiv:1105.3423.
  • [21] Zhou, W. (2007). Asymptotic distribution of the largest off-diagonal entry of correlation matrices. Trans. Amer. Math. Soc. 359 5345–5363.

Supplemental materials

  • Supplement to “Spectral statistics of large dimensional Spearman’s rank correlation matrix and its application”. This supplemental article [5] contains the proofs of Lemmas 3.2, 4.1 4.5, 4.6, 4.11, Propositions 4.2, 4.15, Lemmas 5.1, 6.1, 6.2.