The Annals of Statistics

Limiting laws of coherence of random matrices with applications to testing covariance structure and construction of compressed sensing matrices

T. Tony Cai and Tiefeng Jiang

Full-text: Open access

Abstract

Testing covariance structure is of significant interest in many areas of statistical analysis and construction of compressed sensing matrices is an important problem in signal processing. Motivated by these applications, we study in this paper the limiting laws of the coherence of an n × p random matrix in the high-dimensional setting where p can be much larger than n. Both the law of large numbers and the limiting distribution are derived. We then consider testing the bandedness of the covariance matrix of a high-dimensional Gaussian distribution which includes testing for independence as a special case. The limiting laws of the coherence of the data matrix play a critical role in the construction of the test. We also apply the asymptotic results to the construction of compressed sensing matrices.

Article information

Source
Ann. Statist., Volume 39, Number 3 (2011), 1496-1525.

Dates
First available in Project Euclid: 13 May 2011

Permanent link to this document
https://projecteuclid.org/euclid.aos/1305292044

Digital Object Identifier
doi:10.1214/11-AOS879

Mathematical Reviews number (MathSciNet)
MR2850210

Zentralblatt MATH identifier
1220.62066

Subjects
Primary: 62H12: Estimation 60F05: Central limit and other weak theorems
Secondary: 60F15: Strong theorems 62H10: Distribution of statistics

Keywords
Chen–Stein method coherence compressed sensing matrix covariance structure law of large numbers limiting distribution maxima moderate deviations mutual incoherence property random matrix sample correlation matrix

Citation

Cai, T. Tony; Jiang, Tiefeng. Limiting laws of coherence of random matrices with applications to testing covariance structure and construction of compressed sensing matrices. Ann. Statist. 39 (2011), no. 3, 1496--1525. doi:10.1214/11-AOS879. https://projecteuclid.org/euclid.aos/1305292044


Export citation

References

  • Achlioptas, D. (2003). Database-friendly random projections: Johnson–Lindenstrauss with binary coins. J. Comput. Syst. Sci. 66 671–687.
  • Anderson, G. W., Guionnet, A. and Zeitouni, O. (2009). An Introduction to Random Matrices. Cambridge Univ. Press, Cambridge.
  • Andrews, D. W. K. (1991). Heteroskedasticity and autocorrelation consistent covariance matrix estimation. Econometrica 59 817–858.
  • Arratia, R., Goldstein, L. and Gordon, L. (1989). Two moments suffice for Poisson approximations: The Chen–Stein method. Ann. Probab. 17 9–25.
  • Bai, Z. and Saranadasa, H. (1996). Effect of high dimension: By an example of a two sample problem. Statist. Sinica 6 311–329.
  • Bai, Z. and Silverstein, J. W. (2010). Spectral Analysis of Large Dimensional Random Matrices, 2nd ed. Springer, New York.
  • Bai, Z. D., Miao, B. Q. and Pan, G. M. (2007). On asymptotics of eigenvectors of large sample covariance matrix. Ann. Probab. 35 1532–1572.
  • Bai, Z., Jiang, D., Yao, J.-F. and Zheng, S. (2009). Corrections to LRT on large-dimensional covariance matrix by RMT. Ann. Statist. 37 3822–3840.
  • Baraniuk, R., Davenport, M., DeVore, R. and Wakin, M. (2008). A simple proof of the restricted isometry property for random matrices. Constr. Approx. 28 253–263.
  • Bickel, P. J., Ritov, Y. and Tsybakov, A. B. (2009). Simultaneous analysis of Lasso and Dantzig selector. Ann. Statist. 37 1705–1732.
  • Cai, T. and Jiang, T. (2010). Supplement to “Limiting laws of coherence of random matrices with applications to testing covariance structure and construction of compressed sensing matrices.” DOI:10.1214/11-AOS879SUPP.
  • Cai, T. T. and Lv, J. (2007). Discussion: “The Dantzig selector: Statistical estimation when p is much larger than n,” by E. Candes and T. Tao. Ann. Statist. 35 2365–2369.
  • Cai, T. T., Wang, L. and Xu, G. (2010a). Shifting inequality and recovery of sparse signals. IEEE Trans. Signal Process. 58 1300–1308.
  • Cai, T. T., Wang, L. and Xu, G. (2010b). Stable recovery of sparse signals and an oracle inequality. IEEE Trans. Inform. Theory 56 3516–3522.
  • Cai, T. T., Zhang, C.-H. and Zhou, H. H. (2010). Optimal rates of convergence for covariance matrix estimation. Ann. Statist. 38 2118–2144.
  • Candès, E. J. and Plan, Y. (2009). Near-ideal model selection by 1 minimization. Ann. Statist. 37 2145–2177.
  • Candes, E. J. and Tao, T. (2005). Decoding by linear programming. IEEE Trans. Inform. Theory 51 4203–4215.
  • Candes, E. and Tao, T. (2007). The Dantzig selector: Statistical estimation when p is much larger than n (with discussion). Ann. Statist. 35 2313–2351.
  • Chen, X. (1990). Moderate deviations of B-valued independent random vectors. Chinese Ann. Math. Ser. A 11 621–629.
  • Chen, X. (1991). Moderate deviations of independent random vectors in a Banach space. Chinese J. Appl. Probab. Statist. 7 24–32.
  • Dembo, A. and Zeitouni, O. (1998). Large Deviations Techniques and Applications, 2nd ed. Applications of Mathematics (New York) 38. Springer, New York.
  • Diaconis, P. W., Eaton, M. L. and Lauritzen, S. L. (1992). Finite de Finetti theorems in linear models and multivariate analysis. Scand. J. Statist. 19 289–315.
  • Donoho, D. L. (2006a). Compressed sensing. IEEE Trans. Inform. Theory 52 1289–1306.
  • Donoho, D. L. (2006b). For most large underdetermined systems of linear equations the minimal l1-norm solution is also the sparsest solution. Comm. Pure Appl. Math. 59 797–829.
  • Donoho, D. L. and Huo, X. (2001). Uncertainty principles and ideal atomic decomposition. IEEE Trans. Inform. Theory 47 2845–2862.
  • Donoho, D. L., Elad, M. and Temlyakov, V. N. (2006). Stable recovery of sparse overcomplete representations in the presence of noise. IEEE Trans. Inform. Theory 52 6–18.
  • Fan, J. and Lv, J. (2008). Sure independence screening for ultrahigh dimensional feature space. J. R. Stat. Soc. Ser. B Stat. Methodol. 70 849–911.
  • Fan, J. and Lv, J. (2010). A selective overview of variable selection in high dimensional feature space. Statist. Sinica 20 101–148.
  • Fuchs, J.-J. (2004). On sparse representations in arbitrary redundant bases. IEEE Trans. Inform. Theory 50 1341–1344.
  • Jiang, T. (2004a). The asymptotic distributions of the largest entries of sample correlation matrices. Ann. Appl. Probab. 14 865–880.
  • Jiang, T. (2004b). The limiting distributions of eigenvalues of sample correlation matrices. Sankhyā 66 35–48.
  • Jiang, T. (2005). Maxima of entries of Haar distributed matrices. Probab. Theory Related Fields 131 121–144.
  • Jiang, T. (2006). How many entries of a typical orthogonal matrix can be approximated by independent normals? Ann. Probab. 34 1497–1529.
  • Jiang, T. (2009). The entries of circular orthogonal ensembles. J. Math. Phys. 50 063302, 13.
  • Johnstone, I. M. (2001). On the distribution of the largest eigenvalue in principal components analysis. Ann. Statist. 29 295–327.
  • Johnstone, I. M. (2008). Multivariate analysis and Jacobi ensembles: Largest eigenvalue, Tracy–Widom limits and rates of convergence. Ann. Statist. 36 2638–2716.
  • Ledoux, M. (1992). Sur les déviations modérées des sommes de variables aléatoires vectorielles indépendantes de même loi. Ann. Inst. H. Poincaré Probab. Statist. 28 267–280.
  • Li, D. and Rosalsky, A. (2006). Some strong limit theorems for the largest entries of sample correlation matrices. Ann. Appl. Probab. 16 423–447.
  • Li, D., Liu, W. D. and Rosalsky, A. (2009). Necessary and sufficient conditions for the asymptotic distribution of the largest entry of a sample correlation matrix. Probab. Theory Related Fields 148 5–35.
  • Ligeralde, A. and Brown, B. (1995). Band covariance matrix estimation using restricted residuals: A Monte Carlo analysis. Internat. Econom. Rev. 36 751–767.
  • Liu, W.-D., Lin, Z. and Shao, Q.-M. (2008). The asymptotic distribution and Berry–Esseen bound of a new test for independence in high dimension with an application to stochastic optimization. Ann. Appl. Probab. 18 2337–2366.
  • Péché, S. (2009). Universality results for the largest eigenvalues of some sample covariance matrix ensembles. Probab. Theory Related Fields 143 481–516.
  • Sakhanenko, A. I. (1991). Estimates of Berry–Esseen type for the probabilities of large deviations. Sibirsk. Mat. Zh. 32 133–142, 228.
  • Zhou, W. (2007). Asymptotic distribution of the largest off-diagonal entry of correlation matrices. Trans. Amer. Math. Soc. 359 5345–5363.

Supplemental materials