Electronic Journal of Statistics

The graphical lasso: New insights and alternatives

Rahul Mazumder and Trevor Hastie

Full-text: Access has been disabled (more information)

Abstract

The graphical lasso [5] is an algorithm for learning the structure in an undirected Gaussian graphical model, using $\ell_{1}$ regularization to control the number of zeros in the precision matrix $\boldsymbol{\Theta}=\boldsymbol{\Sigma}^{-1}$ [2, 11]. The R package GLASSO [5] is popular, fast, and allows one to efficiently build a path of models for different values of the tuning parameter. Convergence of GLASSO can be tricky; the converged precision matrix might not be the inverse of the estimated covariance, and occasionally it fails to converge with warm starts. In this paper we explain this behavior, and propose new algorithms that appear to outperform GLASSO.

By studying the “normal equations” we see that, GLASSO is solving the dual of the graphical lasso penalized likelihood, by block coordinate ascent; a result which can also be found in [2]. In this dual, the target of estimation is $\boldsymbol{\Sigma}$, the covariance matrix, rather than the precision matrix $\boldsymbol{\Theta}$. We propose similar primal algorithms P-GLASSO and DP-GLASSO, that also operate by block-coordinate descent, where $\boldsymbol{\Theta}$ is the optimization target. We study all of these algorithms, and in particular different approaches to solving their coordinate sub-problems. We conclude that DP-GLASSO is superior from several points of view.

Article information

Source
Electron. J. Statist. Volume 6 (2012), 2125-2149.

Dates
First available in Project Euclid: 9 November 2012

Permanent link to this document
http://projecteuclid.org/euclid.ejs/1352470831

Digital Object Identifier
doi:10.1214/12-EJS740

Mathematical Reviews number (MathSciNet)
MR3020259

Zentralblatt MATH identifier
1295.62066

Subjects
Primary: 62H99: None of the above, but in this section 62-09: Graphical methods
Secondary: 62-04: Explicit machine computation and programs (not the theory of computation or programming)

Keywords
Graphical lasso sparse inverse covariance selection precision matrix convex analysis/optimization positive definite matrices sparsity semidefinite programming

Citation

Mazumder, Rahul; Hastie, Trevor. The graphical lasso: New insights and alternatives. Electron. J. Statist. 6 (2012), 2125--2149. doi:10.1214/12-EJS740. http://projecteuclid.org/euclid.ejs/1352470831.


Export citation

References

  • [1] Alon, U., Barkai, N., Notterman, D. A., Gish, K., Ybarra, S., Mack, D. and Levine, A. J. (1999). Broad patterns of gene expression revealed by clustering analysis of tumor and normal colon tissues probed by oligonucleotide arrays., Proceedings of the National Academy of Sciences of the United States of America 96 6745–6750.
  • [2] Banerjee, O., Ghaoui, L. E. and d’Aspremont, A. (2008). Model Selection Through Sparse Maximum Likelihood Estimation for Multivariate Gaussian or Binary Data., Journal of Machine Learning Research 9 485–516.
    Mathematical Reviews (MathSciNet): MR2417243
    Zentralblatt MATH: 1225.68149
  • [3] Beck, A. and Teboulle, M. (2009). A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems., SIAM J. Imaging Sciences 2 183-202.
    Mathematical Reviews (MathSciNet): MR2486527
    Zentralblatt MATH: 1175.94009
    Digital Object Identifier: doi:10.1137/080716542
  • [4] Boyd, S. and Vandenberghe, L. (2004)., Convex Optimization. Cambridge University Press.
    Mathematical Reviews (MathSciNet): MR2061575
  • [5] Friedman, J., Hastie, T. and Tibshirani, R. (2007). Sparse inverse covariance estimation with the graphical lasso., Biostatistics 9 432–441.
  • [6] Hastie, T., Tibshirani, R. and Friedman, J. (2009)., The Elements of Statistical Learning, Second Edition: Data Mining, Inference, and Prediction (Springer Series in Statistics), 2 ed. Springer New York.
    Mathematical Reviews (MathSciNet): MR2722294
  • [7] Mazumder, R. and Hastie, T. (2012). Exact Covariance Thresholding into Connected Components for Large-Scale Graphical Lasso., Journal of Machine Learning Research 13 781–794.
    Mathematical Reviews (MathSciNet): MR2913718
  • [8] Meinshausen, N. and Bühlmann, P. (2006). High-dimensional graphs and variable selection with the lasso., Annals of Statistics 34 1436–1462.
    Mathematical Reviews (MathSciNet): MR2278363
    Zentralblatt MATH: 1113.62082
    Digital Object Identifier: doi:10.1214/009053606000000281
    Project Euclid: euclid.aos/1152540754
  • [9] Nesterov, Y. (2007). Gradient methods for minimizing composite objective function Technical Report, Center for Operations Research and Econometrics (CORE), Catholic University of Louvain. Tech. Rep, 76.
  • [10] Rothman, A. J., Bickel, P. J., Levina, E. and Zhu, J. (2008). Sparse Permutation Invariant Covariance Estimation., Electronic Journal of Statistics 2 494-515.
    Mathematical Reviews (MathSciNet): MR2417391
    Digital Object Identifier: doi:10.1214/08-EJS176
    Project Euclid: euclid.ejs/1214491853
  • [11] Yuan, M. and Lin, Y. (2007). Model selection and estimation in the Gaussian graphical model., Biometrika 94 19-35.
    Mathematical Reviews (MathSciNet): MR2367824
    Zentralblatt MATH: 1142.62408
    Digital Object Identifier: doi:10.1093/biomet/asm018

The Institute of Mathematical Statistics and the Bernoulli Society

Electronic Journal of Statistics

New content alerts

Email RSS ToC RSS Article
  • You have access to this content.
  • You have partial access to this content.
  • You do not have access to this content.
Turn Off MathJax

What is MathJax?

More like this

+ See more