The Annals of Statistics

Nonlinear shrinkage estimation of large-dimensional covariance matrices

Olivier Ledoit and Michael Wolf

Full-text: Open access

Abstract

Many statistical applications require an estimate of a covariance matrix and/or its inverse. When the matrix dimension is large compared to the sample size, which happens frequently, the sample covariance matrix is known to perform poorly and may suffer from ill-conditioning. There already exists an extensive literature concerning improved estimators in such situations. In the absence of further knowledge about the structure of the true covariance matrix, the most successful approach so far, arguably, has been shrinkage estimation. Shrinking the sample covariance matrix to a multiple of the identity, by taking a weighted average of the two, turns out to be equivalent to linearly shrinking the sample eigenvalues to their grand mean, while retaining the sample eigenvectors. Our paper extends this approach by considering nonlinear transformations of the sample eigenvalues. We show how to construct an estimator that is asymptotically equivalent to an oracle estimator suggested in previous work. As demonstrated in extensive Monte Carlo simulations, the resulting bona fide estimator can result in sizeable improvements over the sample covariance matrix and also over linear shrinkage.

Article information

Source
Ann. Statist. Volume 40, Number 2 (2012), 1024-1060.

Dates
First available in Project Euclid: 18 July 2012

Permanent link to this document
https://projecteuclid.org/euclid.aos/1342625460

Digital Object Identifier
doi:10.1214/12-AOS989

Mathematical Reviews number (MathSciNet)
MR2985942

Zentralblatt MATH identifier
1274.62371

Subjects
Primary: 62H12: Estimation
Secondary: 62G20: Asymptotic properties 15A52

Keywords
Large-dimensional asymptotics nonlinear shrinkage rotation equivariance

Citation

Ledoit, Olivier; Wolf, Michael. Nonlinear shrinkage estimation of large-dimensional covariance matrices. Ann. Statist. 40 (2012), no. 2, 1024--1060. doi:10.1214/12-AOS989. https://projecteuclid.org/euclid.aos/1342625460


Export citation

References

  • Bai, Z. D. and Silverstein, J. W. (1998). No eigenvalues outside the support of the limiting spectral distribution of large-dimensional sample covariance matrices. Ann. Probab. 26 316–345.
  • Bickel, P. J. and Levina, E. (2008). Regularized estimation of large covariance matrices. Ann. Statist. 36 199–227.
  • Cai, T. and Zhou, H. (2012). Minimax estimation of large covariance matrices under $\ell _1$ norm. Statist. Sinica. To appear.
  • El Karoui, N. (2008). Spectrum estimation for large dimensional covariance matrices using random matrix theory. Ann. Statist. 36 2757–2790.
  • Fan, J., Fan, Y. and Lv, J. (2008). High dimensional covariance matrix estimation using a factor model. J. Econometrics 147 186–197.
  • Gill, P. E., Murray, W. and Saunders, M. A. (2002). SNOPT: An SQP algorithm for large-scale constrained optimization. SIAM J. Optim. 12 979–1006 (electronic).
  • Haff, L. R. (1980). Empirical Bayes estimation of the multivariate normal covariance matrix. Ann. Statist. 8 586–597.
  • James, W. and Stein, C. (1961). Estimation with quadratic loss. In Proc. 4th Berkeley Sympos. Math. Statist. and Prob., Vol. I 361–379. Univ. California Press, Berkeley, Calif.
  • Khare, K. and Rajaratnam, B. (2011). Wishart distributions for decomposable covariance graph models. Ann. Statist. 39 514–555.
  • Ledoit, O. and Péché, S. (2011). Eigenvectors of some large sample covariance matrix ensembles. Probab. Theory Related Fields 151 233–264.
  • Ledoit, O. and Wolf, M. (2004). A well-conditioned estimator for large-dimensional covariance matrices. J. Multivariate Anal. 88 365–411.
  • Ledoit, O. and Wolf, M. (2012). Supplement to “Nonlinear shrinkage estimation of large-dimensional covariance matrices.” DOI:10.1214/12-AOS989SUPP.
  • Marčenko, V. A. and Pastur, L. A. (1967). Distribution of eigenvalues for some sets of random matrices. Sbornik: Mathematics 1 457–483.
  • Mestre, X. and Lagunas, M. A. (2006). Finite sample size effect on minimum variance beamformers: Optimum diagonal loading factor for large arrays. IEEE Trans. Signal Process. 54 69–82.
  • Perlman, M. D. (2007). STAT 542: Multivariate Statistical Analysis. Univ. Washington (On-Line Class Notes), Seattle, Washington.
  • Rajaratnam, B., Massam, H. and Carvalho, C. M. (2008). Flexible covariance estimation in graphical Gaussian models. Ann. Statist. 36 2818–2849.
  • Ravikumar, P., Wawinwright, M., Raskutti, G. and Yu, B. (2008). High-dimensional covariance estimation by minimizing $\ell_1$-penalized log-determinant divergence Technical Report 797, Dept. Statistics, Univ. California, Berkeley.
  • Rohde, A. and Tsybakov, A. B. (2011). Estimation of high-dimensional low-rank matrices. Ann. Statist. 39 887–930.
  • Silverman, B. W. (1986). Density Estimation for Statistics and Data Analysis. Chapman & Hall, London.
  • Silverstein, J. W. (1995). Strong convergence of the empirical distribution of eigenvalues of large-dimensional random matrices. J. Multivariate Anal. 55 331–339.
  • Silverstein, J. W. and Choi, S.-I. (1995). Analysis of the limiting spectral distribution of large-dimensional random matrices. J. Multivariate Anal. 54 295–309.
  • Stein, C. (1956). Inadmissibility of the usual estimator for the mean of a multivariate normal distribution. In Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability, 19541955, Vol. I 197–206. Univ. California Press, Berkeley.
  • Stein, C. (1975). Estimation of a covariance matrix. Rietz lecture, 39th Annual Meeting IMS. Atlanta, Georgia.
  • Won, J. H., Lim, J., Kim, S. J. and Rajaratnam, B. (2009). Maximum likelihood covariance estimation with a condition number constraint. Technical Report 2009-10, Dept. Statistics, Stanford Univ.

Supplemental materials