Annals of Statistics

Nonlinear shrinkage estimation of large-dimensional covariance matrices

Olivier Ledoit and Michael Wolf

Full-text: Open access

Enhanced PDF (576 KB)

Abstract

Many statistical applications require an estimate of a covariance matrix and/or its inverse. When the matrix dimension is large compared to the sample size, which happens frequently, the sample covariance matrix is known to perform poorly and may suffer from ill-conditioning. There already exists an extensive literature concerning improved estimators in such situations. In the absence of further knowledge about the structure of the true covariance matrix, the most successful approach so far, arguably, has been shrinkage estimation. Shrinking the sample covariance matrix to a multiple of the identity, by taking a weighted average of the two, turns out to be equivalent to linearly shrinking the sample eigenvalues to their grand mean, while retaining the sample eigenvectors. Our paper extends this approach by considering nonlinear transformations of the sample eigenvalues. We show how to construct an estimator that is asymptotically equivalent to an oracle estimator suggested in previous work. As demonstrated in extensive Monte Carlo simulations, the resulting bona fide estimator can result in sizeable improvements over the sample covariance matrix and also over linear shrinkage.

Article information

Source
Ann. Statist., Volume 40, Number 2 (2012), 1024-1060.

Dates
First available in Project Euclid: 18 July 2012

Permanent link to this document
https://projecteuclid.org/euclid.aos/1342625460

Digital Object Identifier
doi:10.1214/12-AOS989

Mathematical Reviews number (MathSciNet)
MR2985942

Zentralblatt MATH identifier
1274.62371

Subjects
Primary: 62H12: Estimation
Secondary: 62G20: Asymptotic properties 15A52

Keywords
Large-dimensional asymptotics nonlinear shrinkage rotation equivariance

Citation

Ledoit, Olivier; Wolf, Michael. Nonlinear shrinkage estimation of large-dimensional covariance matrices. Ann. Statist. 40 (2012), no. 2, 1024--1060. doi:10.1214/12-AOS989. https://projecteuclid.org/euclid.aos/1342625460


Export citation

References

  • Bai, Z. D. and Silverstein, J. W. (1998). No eigenvalues outside the support of the limiting spectral distribution of large-dimensional sample covariance matrices. Ann. Probab. 26 316–345.
    Mathematical Reviews (MathSciNet): MR1617051
    Zentralblatt MATH: 0937.60017
    Digital Object Identifier: doi:10.1214/aop/1022855421
    Project Euclid: euclid.aop/1022855421
  • Bickel, P. J. and Levina, E. (2008). Regularized estimation of large covariance matrices. Ann. Statist. 36 199–227.
    Mathematical Reviews (MathSciNet): MR2387969
    Zentralblatt MATH: 1132.62040
    Digital Object Identifier: doi:10.1214/009053607000000758
    Project Euclid: euclid.aos/1201877299
  • Cai, T. and Zhou, H. (2012). Minimax estimation of large covariance matrices under $\ell _1$ norm. Statist. Sinica. To appear.
  • El Karoui, N. (2008). Spectrum estimation for large dimensional covariance matrices using random matrix theory. Ann. Statist. 36 2757–2790.
    Mathematical Reviews (MathSciNet): MR2485012
    Zentralblatt MATH: 1168.62052
    Digital Object Identifier: doi:10.1214/07-AOS581
    Project Euclid: euclid.aos/1231165184
  • Fan, J., Fan, Y. and Lv, J. (2008). High dimensional covariance matrix estimation using a factor model. J. Econometrics 147 186–197.
    Mathematical Reviews (MathSciNet): MR2472991
    Digital Object Identifier: doi:10.1016/j.jeconom.2008.09.017
  • Gill, P. E., Murray, W. and Saunders, M. A. (2002). SNOPT: An SQP algorithm for large-scale constrained optimization. SIAM J. Optim. 12 979–1006 (electronic).
    Mathematical Reviews (MathSciNet): MR1922505
    Zentralblatt MATH: 1027.90111
    Digital Object Identifier: doi:10.1137/S1052623499350013
  • Haff, L. R. (1980). Empirical Bayes estimation of the multivariate normal covariance matrix. Ann. Statist. 8 586–597.
    Mathematical Reviews (MathSciNet): MR568722
    Zentralblatt MATH: 0441.62045
    Digital Object Identifier: doi:10.1214/aos/1176345010
    Project Euclid: euclid.aos/1176345010
  • James, W. and Stein, C. (1961). Estimation with quadratic loss. In Proc. 4th Berkeley Sympos. Math. Statist. and Prob., Vol. I 361–379. Univ. California Press, Berkeley, Calif.
    Mathematical Reviews (MathSciNet): MR133191
  • Khare, K. and Rajaratnam, B. (2011). Wishart distributions for decomposable covariance graph models. Ann. Statist. 39 514–555.
    Mathematical Reviews (MathSciNet): MR2797855
    Zentralblatt MATH: 05874504
    Digital Object Identifier: doi:10.1214/10-AOS841
    Project Euclid: euclid.aos/1297779855
  • Ledoit, O. and Péché, S. (2011). Eigenvectors of some large sample covariance matrix ensembles. Probab. Theory Related Fields 151 233–264.
    Mathematical Reviews (MathSciNet): MR2834718
    Zentralblatt MATH: 1229.60009
    Digital Object Identifier: doi:10.1007/s00440-010-0298-3
  • Ledoit, O. and Wolf, M. (2004). A well-conditioned estimator for large-dimensional covariance matrices. J. Multivariate Anal. 88 365–411.
    Mathematical Reviews (MathSciNet): MR2026339
    Zentralblatt MATH: 1032.62050
    Digital Object Identifier: doi:10.1016/S0047-259X(03)00096-4
  • Ledoit, O. and Wolf, M. (2012). Supplement to “Nonlinear shrinkage estimation of large-dimensional covariance matrices.” DOI:10.1214/12-AOS989SUPP.
  • Marčenko, V. A. and Pastur, L. A. (1967). Distribution of eigenvalues for some sets of random matrices. Sbornik: Mathematics 1 457–483.
  • Mestre, X. and Lagunas, M. A. (2006). Finite sample size effect on minimum variance beamformers: Optimum diagonal loading factor for large arrays. IEEE Trans. Signal Process. 54 69–82.
  • Perlman, M. D. (2007). STAT 542: Multivariate Statistical Analysis. Univ. Washington (On-Line Class Notes), Seattle, Washington.
  • Rajaratnam, B., Massam, H. and Carvalho, C. M. (2008). Flexible covariance estimation in graphical Gaussian models. Ann. Statist. 36 2818–2849.
    Mathematical Reviews (MathSciNet): MR2485014
    Zentralblatt MATH: 1168.62054
    Digital Object Identifier: doi:10.1214/08-AOS619
    Project Euclid: euclid.aos/1231165186
  • Ravikumar, P., Wawinwright, M., Raskutti, G. and Yu, B. (2008). High-dimensional covariance estimation by minimizing $\ell_1$-penalized log-determinant divergence Technical Report 797, Dept. Statistics, Univ. California, Berkeley.
  • Rohde, A. and Tsybakov, A. B. (2011). Estimation of high-dimensional low-rank matrices. Ann. Statist. 39 887–930.
    Mathematical Reviews (MathSciNet): MR2816342
    Zentralblatt MATH: 1215.62056
    Digital Object Identifier: doi:10.1214/10-AOS860
    Project Euclid: euclid.aos/1299680958
  • Silverman, B. W. (1986). Density Estimation for Statistics and Data Analysis. Chapman & Hall, London.
    Mathematical Reviews (MathSciNet): MR848134
    Zentralblatt MATH: 0617.62042
  • Silverstein, J. W. (1995). Strong convergence of the empirical distribution of eigenvalues of large-dimensional random matrices. J. Multivariate Anal. 55 331–339.
    Mathematical Reviews (MathSciNet): MR1370408
    Zentralblatt MATH: 0851.62015
    Digital Object Identifier: doi:10.1006/jmva.1995.1083
  • Silverstein, J. W. and Choi, S.-I. (1995). Analysis of the limiting spectral distribution of large-dimensional random matrices. J. Multivariate Anal. 54 295–309.
    Mathematical Reviews (MathSciNet): MR1345541
    Zentralblatt MATH: 0872.60013
    Digital Object Identifier: doi:10.1006/jmva.1995.1058
  • Stein, C. (1956). Inadmissibility of the usual estimator for the mean of a multivariate normal distribution. In Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability, 1954–1955, Vol. I 197–206. Univ. California Press, Berkeley.
    Mathematical Reviews (MathSciNet): MR84922
    Zentralblatt MATH: 0073.35602
  • Stein, C. (1975). Estimation of a covariance matrix. Rietz lecture, 39th Annual Meeting IMS. Atlanta, Georgia.
  • Won, J. H., Lim, J., Kim, S. J. and Rajaratnam, B. (2009). Maximum likelihood covariance estimation with a condition number constraint. Technical Report 2009-10, Dept. Statistics, Stanford Univ.

Supplemental materials

The Institute of Mathematical Statistics

Annals of Statistics

New content alerts

Email RSS ToC RSS Article
  • You have access to this content.
  • You have partial access to this content.
  • You do not have access to this content.
Turn Off MathJax

What is MathJax?

More like this

+ See more