## Bernoulli

• Bernoulli
• Volume 24, Number 4B (2018), 3791-3832.

### Optimal estimation of a large-dimensional covariance matrix under Stein’s loss

#### Abstract

This paper introduces a new method for deriving covariance matrix estimators that are decision-theoretically optimal within a class of nonlinear shrinkage estimators. The key is to employ large-dimensional asymptotics: the matrix dimension and the sample size go to infinity together, with their ratio converging to a finite, nonzero limit. As the main focus, we apply this method to Stein’s loss. Compared to the estimator of Stein (Estimation of a covariance matrix (1975); J. Math. Sci. 34 (1986) 1373–1403), ours has five theoretical advantages: (1) it asymptotically minimizes the loss itself, instead of an estimator of the expected loss; (2) it does not necessitate post-processing via an ad hoc algorithm (called “isotonization”) to restore the positivity or the ordering of the covariance matrix eigenvalues; (3) it does not ignore any terms in the function to be minimized; (4) it does not require normality; and (5) it is not limited to applications where the sample size exceeds the dimension. In addition to these theoretical advantages, our estimator also improves upon Stein’s estimator in terms of finite-sample performance, as evidenced via extensive Monte Carlo simulations. To further demonstrate the effectiveness of our method, we show that some previously suggested estimators of the covariance matrix and its inverse are decision-theoretically optimal in the large-dimensional asymptotic limit with respect to the Frobenius loss function.

#### Article information

Source
Bernoulli, Volume 24, Number 4B (2018), 3791-3832.

Dates
Revised: July 2017
First available in Project Euclid: 18 April 2018

https://projecteuclid.org/euclid.bj/1524038770

Digital Object Identifier
doi:10.3150/17-BEJ979

Mathematical Reviews number (MathSciNet)
MR3788189

Zentralblatt MATH identifier
06869892

#### Citation

Ledoit, Olivier; Wolf, Michael. Optimal estimation of a large-dimensional covariance matrix under Stein’s loss. Bernoulli 24 (2018), no. 4B, 3791--3832. doi:10.3150/17-BEJ979. https://projecteuclid.org/euclid.bj/1524038770

#### References

• [1] Bai, Z. and Silverstein, J.W. (2010). Spectral Analysis of Large Dimensional Random Matrices, 2nd ed. Springer Series in Statistics. New York: Springer.
• [2] Bai, Z.D. and Silverstein, J.W. (1998). No eigenvalues outside the support of the limiting spectral distribution of large-dimensional sample covariance matrices. Ann. Probab. 26 316–345.
• [3] Bai, Z.D. and Silverstein, J.W. (1999). Exact separation of eigenvalues of large-dimensional sample covariance matrices. Ann. Probab. 27 1536–1555.
• [4] Bai, Z.D. and Silverstein, J.W. (2004). CLT for linear spectral statistics of large-dimensional sample covariance matrices. Ann. Probab. 32 553–605.
• [5] Bai, Z.D., Silverstein, J.W. and Yin, Y.Q. (1988). A note on the largest eigenvalue of a large-dimensional sample covariance matrix. J. Multivariate Anal. 26 166–168.
• [6] Baik, J., Ben Arous, G. and Péché, S. (2005). Phase transition of the largest eigenvalue for nonnull complex sample covariance matrices. Ann. Probab. 33 1643–1697.
• [7] Baik, J. and Silverstein, J.W. (2006). Eigenvalues of large sample covariance matrices of spiked population models. J. Multivariate Anal. 97 1382–1408.
• [8] Bickel, P.J. and Levina, E. (2008). Covariance regularization by thresholding. Ann. Statist. 36 2577–2604.
• [9] Chen, Y., Wiesel, A. and Hero, A.O. (2009). Shrinkage estimation of high dimensional covariance matrices. In IEEE International Conference on Acoustics, Speech, and Signal Processing, Taiwan.
• [10] Daniels, M.J. and Kass, R.E. (2001). Shrinkage estimators for covariance matrices. Biometrics 57 1173–1184.
• [11] DeMiguel, V., Garlappi, L. and Uppal, R. (2009). Optimal versus naive diversification: How inefficient is the $1/N$ portfolio strategy? Rev. Financ. Stud. 22 1915–1953.
• [12] Dey, D.K. and Srinivasan, C. (1985). Estimation of a covariance matrix under Stein’s loss. Ann. Statist. 13 1581–1591.
• [13] Donoho, D.L., Gavish, M. and Johnstone, I.M. (2014). Optimal shrinkage of eigenvalues in the spiked covariance model. arXiv:1311.0851v2.
• [14] El Karoui, N. (2008). Spectrum estimation for large dimensional covariance matrices using random matrix theory. Ann. Statist. 36 2757–2790.
• [15] Gill, P.E., Murray, W. and Saunders, M.A. (2002). SNOPT: An SQP algorithm for large-scale constrained optimization. SIAM J. Optim. 12 979–1006.
• [16] Haff, L.R. (1980). Empirical Bayes estimation of the multivariate normal covariance matrix. Ann. Statist. 8 586–597.
• [17] Haugen, R.A. and Baker, N.L. (1991). The efficient market inefficiency of capitalization-weighted stock portfolios. J. Portf. Manag. 17 35–40.
• [18] Henrici, P. (1988). Applied and Computational Complex Analysis 1. New York: Wiley.
• [19] Jagannathan, R. and Ma, T. (2003). Risk reduction in large portfolios: Why imposing the wrong constraints helps. J. Finance 54 1651–1684.
• [20] James, W. and Stein, C. (1961). Estimation with quadratic loss. In Proc. 4th Berkeley Sympos. Math. Statist. and Prob., Vol. I 361–379. Berkeley, CA: Univ. California Press.
• [21] Jeffreys, H. (1946). An invariant form for the prior probability in estimation problems. Proc. R. Soc. Lond. Ser. A Math. Phys. Eng. Sci. 186 453–461.
• [22] Johnstone, I.M. (2001). On the distribution of the largest eigenvalue in principal components analysis. Ann. Statist. 29 295–327.
• [23] Khare, K., Oh, S.-Y. and Rajaratnam, B. (2015). A convex pseudolikelihood framework for high dimensional partial correlation estimation with convergence guarantees. J. R. Stat. Soc. Ser. B. Stat. Methodol. 77 803–825.
• [24] Kullback, S. and Leibler, R.A. (1951). On information and sufficiency. Ann. Math. Stat. 22 79–86.
• [25] Ledoit, O. and Péché, S. (2011). Eigenvectors of some large sample covariance matrix ensembles. Probab. Theory Related Fields 151 233–264.
• [26] Ledoit, O. and Wolf, M. (2004). A well-conditioned estimator for large-dimensional covariance matrices. J. Multivariate Anal. 88 365–411.
• [27] Ledoit, O. and Wolf, M. (2012). Nonlinear shrinkage estimation of large-dimensional covariance matrices. Ann. Statist. 40 1024–1060.
• [28] Ledoit, O. and Wolf, M. (2015). Spectrum estimation: A unified framework for covariance matrix estimation and PCA in large dimensions. J. Multivariate Anal. 139 360–384.
• [29] Ledoit, O. and Wolf, M. (2017). Supplement to “Optimal estimation of a large-dimensional covariance matrix under Stein’s loss”. DOI:10.3150/17-BEJ979SUPP.
• [30] Lin, S.P. and Perlman, M.D. (1985). A Monte Carlo comparison of four estimators of a covariance matrix. In Multivariate Analysis VI (Pittsburgh, Pa., 1983) 411–429. Amsterdam: North-Holland.
• [31] Marčenko, V.A. and Pastur, L.A. (1967). Distribution of eigenvalues for some sets of random matrices. Sb. Math. 1 457–483.
• [32] Mestre, X. (2008). Improved estimation of eigenvalues and eigenvectors of covariance matrices using their sample estimates. IEEE Trans. Inform. Theory 54 5113–5129.
• [33] Moakher, M. and Batchelor, P.G. (2006). Symmetric positive-definite matrices: From geometry to applications and visualization. In Visualization and Processing of Tensor Fields. Math. Vis. 285–298, 452. Berlin: Springer.
• [34] Muirhead, R.J. (1982). Aspects of Multivariate Statistical Theory. Wiley Series in Probability and Mathematical Statistics. New York: John Wiley & Sons, Inc.
• [35] Nielsen, F. and Aylursubramanian, R. (2008). Far From the Madding Crowd – Volatility Efficient Indices. Research Insights, MSCI Barra.
• [36] Rajaratnam, B. and Vincenzi, D. (2016). A theoretical study of Stein’s covariance estimator. Biometrika 103 653–666.
• [37] Silverstein, J.W. (1995). Strong convergence of the empirical distribution of eigenvalues of large-dimensional random matrices. J. Multivariate Anal. 55 331–339.
• [38] Silverstein, J.W. and Bai, Z.D. (1995). On the empirical distribution of eigenvalues of a class of large-dimensional random matrices. J. Multivariate Anal. 54 175–192.
• [39] Silverstein, J.W. and Choi, S.-I. (1995). Analysis of the limiting spectral distribution of large-dimensional random matrices. J. Multivariate Anal. 54 295–309.
• [40] Stein, C. (1956). Some problems in multivariate analysis, Part I. Technical Report No. 6, Department of Statistics, Stanford Univ.
• [41] Stein, C. (1975). Estimation of a covariance matrix. Rietz lecture, 39th Annual Meeting IMS. Atlanta, Georgia.
• [42] Stein, C. (1986). Lectures on the theory of estimation of many parameters. J. Math. Sci. 34 1373–1403.
• [43] Tsukuma, H. (2005). Estimating the inverse matrix of scale parameters in an elliptically contoured distribution. J. Japan Statist. Soc. 35 21–39.
• [44] Won, J.-H., Lim, J., Kim, S.-J. and Rajaratnam, B. (2013). Condition-number-regularized covariance estimation. J. R. Stat. Soc. Ser. B. Stat. Methodol. 75 427–450.
• [45] Yin, Y.Q., Bai, Z.D. and Krishnaiah, P.R. (1988). On the limit of the largest eigenvalue of the large-dimensional sample covariance matrix. Probab. Theory Related Fields 78 509–521.

#### Supplemental materials

• Supplement: Proofs of mathematical results. This supplement collects the proofs of all mathematical results.