Open Access
August 2011 Covariance Estimation: The GLM and Regularization Perspectives
Mohsen Pourahmadi
Statist. Sci. 26(3): 369-387 (August 2011). DOI: 10.1214/11-STS358


Finding an unconstrained and statistically interpretable reparameterization of a covariance matrix is still an open problem in statistics. Its solution is of central importance in covariance estimation, particularly in the recent high-dimensional data environment where enforcing the positive-definiteness constraint could be computationally expensive. We provide a survey of the progress made in modeling covariance matrices from two relatively complementary perspectives: (1) generalized linear models (GLM) or parsimony and use of covariates in low dimensions, and (2) regularization or sparsity for high-dimensional data. An emerging, unifying and powerful trend in both perspectives is that of reducing a covariance estimation problem to that of estimating a sequence of regression problems. We point out several instances of the regression-based formulation. A notable case is in sparse estimation of a precision matrix or a Gaussian graphical model leading to the fast graphical LASSO algorithm. Some advantages and limitations of the regression-based Cholesky decomposition relative to the classical spectral (eigenvalue) and variance-correlation decompositions are highlighted. The former provides an unconstrained and statistically interpretable reparameterization, and guarantees the positive-definiteness of the estimated covariance matrix. It reduces the unintuitive task of covariance estimation to that of modeling a sequence of regressions at the cost of imposing an a priori order among the variables. Elementwise regularization of the sample covariance matrix such as banding, tapering and thresholding has desirable asymptotic properties and the sparse estimated covariance matrix is positive definite with probability tending to one for large samples and dimensions.


Download Citation

Mohsen Pourahmadi. "Covariance Estimation: The GLM and Regularization Perspectives." Statist. Sci. 26 (3) 369 - 387, August 2011.


Published: August 2011
First available in Project Euclid: 31 October 2011

zbMATH: 1246.62139
MathSciNet: MR2917961
Digital Object Identifier: 10.1214/11-STS358

Keywords: Bayesian estimation , Cholesky decomposition , dependence and correlation , graphical models , longitudinal data , parsimony , penalized likelihood , precision matrix , Sparsity , spectral decomposition , variance-correlation decomposition

Rights: Copyright © 2011 Institute of Mathematical Statistics

Vol.26 • No. 3 • August 2011
Back to Top