### Nonlinear shrinkage estimation of large-dimensional covariance matrices

Olivier Ledoit and Michael Wolf
Source: Ann. Statist. Volume 40, Number 2 (2012), 1024-1060.

#### Abstract

Many statistical applications require an estimate of a covariance matrix and/or its inverse. When the matrix dimension is large compared to the sample size, which happens frequently, the sample covariance matrix is known to perform poorly and may suffer from ill-conditioning. There already exists an extensive literature concerning improved estimators in such situations. In the absence of further knowledge about the structure of the true covariance matrix, the most successful approach so far, arguably, has been shrinkage estimation. Shrinking the sample covariance matrix to a multiple of the identity, by taking a weighted average of the two, turns out to be equivalent to linearly shrinking the sample eigenvalues to their grand mean, while retaining the sample eigenvectors. Our paper extends this approach by considering nonlinear transformations of the sample eigenvalues. We show how to construct an estimator that is asymptotically equivalent to an oracle estimator suggested in previous work. As demonstrated in extensive Monte Carlo simulations, the resulting bona fide estimator can result in sizeable improvements over the sample covariance matrix and also over linear shrinkage.

First Page:

#### Related Works:

Primary Subjects: 62H12
Secondary Subjects: 62G20, 15A52
We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber.

Permanent link to this document: http://projecteuclid.org/euclid.aos/1342625460
Digital Object Identifier: doi:10.1214/12-AOS989
Zentralblatt MATH identifier: 06073784
Mathematical Reviews number (MathSciNet): MR2985942

### References

Bai, Z. D. and Silverstein, J. W. (1998). No eigenvalues outside the support of the limiting spectral distribution of large-dimensional sample covariance matrices. Ann. Probab. 26 316–345.
Mathematical Reviews (MathSciNet): MR1617051
Zentralblatt MATH: 0937.60017
Digital Object Identifier: doi:10.1214/aop/1022855421
Project Euclid: euclid.aop/1022855421
Bickel, P. J. and Levina, E. (2008). Regularized estimation of large covariance matrices. Ann. Statist. 36 199–227.
Mathematical Reviews (MathSciNet): MR2387969
Zentralblatt MATH: 1132.62040
Digital Object Identifier: doi:10.1214/009053607000000758
Project Euclid: euclid.aos/1201877299
Cai, T. and Zhou, H. (2012). Minimax estimation of large covariance matrices under $\ell _1$ norm. Statist. Sinica. To appear.
El Karoui, N. (2008). Spectrum estimation for large dimensional covariance matrices using random matrix theory. Ann. Statist. 36 2757–2790.
Mathematical Reviews (MathSciNet): MR2485012
Zentralblatt MATH: 1168.62052
Digital Object Identifier: doi:10.1214/07-AOS581
Project Euclid: euclid.aos/1231165184
Fan, J., Fan, Y. and Lv, J. (2008). High dimensional covariance matrix estimation using a factor model. J. Econometrics 147 186–197.
Mathematical Reviews (MathSciNet): MR2472991
Digital Object Identifier: doi:10.1016/j.jeconom.2008.09.017
Gill, P. E., Murray, W. and Saunders, M. A. (2002). SNOPT: An SQP algorithm for large-scale constrained optimization. SIAM J. Optim. 12 979–1006 (electronic).
Mathematical Reviews (MathSciNet): MR1922505
Zentralblatt MATH: 1027.90111
Digital Object Identifier: doi:10.1137/S1052623499350013
Haff, L. R. (1980). Empirical Bayes estimation of the multivariate normal covariance matrix. Ann. Statist. 8 586–597.
Mathematical Reviews (MathSciNet): MR568722
Zentralblatt MATH: 0441.62045
Digital Object Identifier: doi:10.1214/aos/1176345010
Project Euclid: euclid.aos/1176345010
James, W. and Stein, C. (1961). Estimation with quadratic loss. In Proc. 4th Berkeley Sympos. Math. Statist. and Prob., Vol. I 361–379. Univ. California Press, Berkeley, Calif.
Mathematical Reviews (MathSciNet): MR133191
Khare, K. and Rajaratnam, B. (2011). Wishart distributions for decomposable covariance graph models. Ann. Statist. 39 514–555.
Mathematical Reviews (MathSciNet): MR2797855
Zentralblatt MATH: 05874504
Digital Object Identifier: doi:10.1214/10-AOS841
Project Euclid: euclid.aos/1297779855
Ledoit, O. and Péché, S. (2011). Eigenvectors of some large sample covariance matrix ensembles. Probab. Theory Related Fields 151 233–264.
Mathematical Reviews (MathSciNet): MR2834718
Zentralblatt MATH: 1229.60009
Digital Object Identifier: doi:10.1007/s00440-010-0298-3
Ledoit, O. and Wolf, M. (2004). A well-conditioned estimator for large-dimensional covariance matrices. J. Multivariate Anal. 88 365–411.
Mathematical Reviews (MathSciNet): MR2026339
Zentralblatt MATH: 1032.62050
Digital Object Identifier: doi:10.1016/S0047-259X(03)00096-4
Ledoit, O. and Wolf, M. (2012). Supplement to “Nonlinear shrinkage estimation of large-dimensional covariance matrices.” DOI:10.1214/12-AOS989SUPP.
Marčenko, V. A. and Pastur, L. A. (1967). Distribution of eigenvalues for some sets of random matrices. Sbornik: Mathematics 1 457–483.
Mestre, X. and Lagunas, M. A. (2006). Finite sample size effect on minimum variance beamformers: Optimum diagonal loading factor for large arrays. IEEE Trans. Signal Process. 54 69–82.
Perlman, M. D. (2007). STAT 542: Multivariate Statistical Analysis. Univ. Washington (On-Line Class Notes), Seattle, Washington.
Rajaratnam, B., Massam, H. and Carvalho, C. M. (2008). Flexible covariance estimation in graphical Gaussian models. Ann. Statist. 36 2818–2849.
Mathematical Reviews (MathSciNet): MR2485014
Zentralblatt MATH: 1168.62054
Digital Object Identifier: doi:10.1214/08-AOS619
Project Euclid: euclid.aos/1231165186
Ravikumar, P., Wawinwright, M., Raskutti, G. and Yu, B. (2008). High-dimensional covariance estimation by minimizing $\ell_1$-penalized log-determinant divergence Technical Report 797, Dept. Statistics, Univ. California, Berkeley.
Rohde, A. and Tsybakov, A. B. (2011). Estimation of high-dimensional low-rank matrices. Ann. Statist. 39 887–930.
Mathematical Reviews (MathSciNet): MR2816342
Zentralblatt MATH: 1215.62056
Digital Object Identifier: doi:10.1214/10-AOS860
Project Euclid: euclid.aos/1299680958
Silverman, B. W. (1986). Density Estimation for Statistics and Data Analysis. Chapman & Hall, London.
Mathematical Reviews (MathSciNet): MR848134
Zentralblatt MATH: 0617.62042
Silverstein, J. W. (1995). Strong convergence of the empirical distribution of eigenvalues of large-dimensional random matrices. J. Multivariate Anal. 55 331–339.
Mathematical Reviews (MathSciNet): MR1370408
Zentralblatt MATH: 0851.62015
Digital Object Identifier: doi:10.1006/jmva.1995.1083
Silverstein, J. W. and Choi, S.-I. (1995). Analysis of the limiting spectral distribution of large-dimensional random matrices. J. Multivariate Anal. 54 295–309.
Mathematical Reviews (MathSciNet): MR1345541
Zentralblatt MATH: 0872.60013
Digital Object Identifier: doi:10.1006/jmva.1995.1058
Stein, C. (1956). Inadmissibility of the usual estimator for the mean of a multivariate normal distribution. In Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability, 19541955, Vol. I 197–206. Univ. California Press, Berkeley.
Mathematical Reviews (MathSciNet): MR84922
Zentralblatt MATH: 0073.35602
Stein, C. (1975). Estimation of a covariance matrix. Rietz lecture, 39th Annual Meeting IMS. Atlanta, Georgia.
Won, J. H., Lim, J., Kim, S. J. and Rajaratnam, B. (2009). Maximum likelihood covariance estimation with a condition number constraint. Technical Report 2009-10, Dept. Statistics, Stanford Univ.