Statistical Science

Sparse Nonparametric Graphical Models

John Lafferty, Han Liu, and Larry Wasserman

Full-text: Open access

Abstract

We present some nonparametric methods for graphical modeling. In the discrete case, where the data are binary or drawn from a finite alphabet, Markov random fields are already essentially nonparametric, since the cliques can take only a finite number of values. Continuous data are different. The Gaussian graphical model is the standard parametric model for continuous data, but it makes distributional assumptions that are often unrealistic. We discuss two approaches to building more flexible graphical models. One allows arbitrary graphs and a nonparametric extension of the Gaussian; the other uses kernel density estimation and restricts the graphs to trees and forests. Examples of both methods are presented. We also discuss possible future research directions for nonparametric graphical modeling.

Article information

Source
Statist. Sci., Volume 27, Number 4 (2012), 519-537.

Dates
First available in Project Euclid: 21 December 2012

Permanent link to this document
https://projecteuclid.org/euclid.ss/1356098554

Digital Object Identifier
doi:10.1214/12-STS391

Mathematical Reviews number (MathSciNet)
MR3025132

Zentralblatt MATH identifier
1331.62219

Keywords
Kernel density estimation Gaussian copula high-dimensional inference undirected graphical model oracle inequality consistency

Citation

Lafferty, John; Liu, Han; Wasserman, Larry. Sparse Nonparametric Graphical Models. Statist. Sci. 27 (2012), no. 4, 519--537. doi:10.1214/12-STS391. https://projecteuclid.org/euclid.ss/1356098554


Export citation

References

  • Banerjee, O., El Ghaoui, L. and d’Aspremont, A. (2008). Model selection through sparse maximum likelihood estimation for multivariate Gaussian or binary data. J. Mach. Learn. Res. 9 485–516.
  • Chandrasekaran, V., Parrilo, P. A. and Willsky, A. S. (2010). Latent variable graphical model selection via convex optimization. Available at http://arxiv.org/abs/1008.1290.
  • Chen, X., Liu, Y., Liu, H. and Carbonell, J. G. (2010). Learning spatial–temporal varying graphs with applications to climate data analysis. In AAAI-10: Twenty-Fourth Conference on Artificial Intelligence (AAAI) AAAI Press, Menlo Park, CA.
  • Choi, M. J., Tan, V. Y., Anandkumar, A. and Willsky, A. S. (2010). Learning latent tree graphical models. Available at http://arxiv.org/abs/1009.2722.
  • Chow, C. and Liu, C. (1968). Approximating discrete probability distributions with dependence trees. IEEE Transactions on Information Theory 14 462–467.
  • Friedman, J. H., Hastie, T. and Tibshirani, R. (2007). Sparse inverse covariance estimation with the graphical lasso. Biostatistics 9 432–441.
  • Giné, E. and Guillou, A. (2002). Rates of strong uniform consistency for multivariate kernel density estimators. Ann. Inst. H. Poincaré Probab. Stat. 38 907–921.
  • Jeon, Y. and Lin, Y. (2006). An effective method for high-dimensional log-density ANOVA estimation, with application to nonparametric graphical model building. Statist. Sinica 16 353–374.
  • Kolar, M., Song, L., Ahmed, A. and Xing, E. P. (2010). Estimating time-varying networks. Ann. Appl. Stat. 4 94–123.
  • Kruskal, J. B. Jr. (1956). On the shortest spanning subtree of a graph and the traveling salesman problem. Proc. Amer. Math. Soc. 7 48–50.
  • Lafferty, J. and Wasserman, L. (2008). Rodeo: Sparse, greedy nonparametric regression. Ann. Statist. 36 28–63.
  • Liu, H., Lafferty, J. and Wasserman, L. (2009). The nonparanormal: Semiparametric estimation of high dimensional undirected graphs. J. Mach. Learn. Res. 10 2295–2328.
  • Liu, H., Chen, X., Lafferty, J. and Wasserman, L. (2010). Graph-valued regression. In Proceedings of the Twenty-Third Annual Conference on Neural Information Processing Systems (NIPS).
  • Liu, H., Xu, M., Gu, H., Gupta, A., Lafferty, J. and Wasserman, L. (2011). Forest density estimation. J. Mach. Learn. Res. 12 907–951.
  • Liu, H., Han, F., Yuan, M., Lafferty, J. and Wasserman, L. (2012). High dimensional semiparametric Gaussian copula graphical models. In Proceedings of the 29th International Conference on Machine Learning (ICML-12). ACM, New York.
  • Mallows, C. L., ed. (1990). The Collected Works of John W. Tukey. Vol. VI: More Mathematical: 19381984. Wadsworth & Brooks/Cole Advanced Books & Software, Pacific Grove, CA.
  • Meinshausen, N. and Bühlmann, P. (2006). High-dimensional graphs and variable selection with the lasso. Ann. Statist. 34 1436–1462.
  • Ravikumar, P., Wainwright, M. J. and Lafferty, J. D. (2010). High-dimensional Ising model selection using $\ell _{1}$-regularized logistic regression. Ann. Statist. 38 1287–1319.
  • Ravikumar, P., Wainwright, M., Raskutti, G. and Yu, B. (2009). Model selection in Gaussian graphical models: High-dimensional consistency of $\ell _{1}$-regularized MLE. In Advances in Neural Information Processing Systems, 22 MIT Press, Cambridge, MA.
  • Rothman, A. J., Bickel, P. J., Levina, E. and Zhu, J. (2008). Sparse permutation invariant covariance estimation. Electron. J. Stat. 2 494–515.
  • Schwaighofer, A., Dejori, M., Tresp, V. and Stetter, M. (2007). Structure learning with nonparametric decomposable models. In Proceedings of the 17th International Conference on Artificial Neural Networks. ICANN’07. Elsevier, New York.
  • Sklar, M. (1959). Fonctions de répartition à $n$ dimensions et leurs marges. Publ. Inst. Statist. Univ. Paris 8 229–231.
  • Wasserman, L. (2006). All of Nonparametric Statistics. Springer, New York.
  • Wille, A., Zimmermann, P., Vranová, E., Fürholz, A., Laule, O., Bleuler, S., Hennig, L., Prelić, A., von Rohr, P., Thiele, L., Zitzler, E., Gruissem, W. and Bühlmann, P. (2004). Sparse Gaussian graphical modelling of the isoprenoid gene network in Arabidopsis thaliana. Genome Biology 5 R92.
  • Yuan, M. and Lin, Y. (2007). Model selection and estimation in the Gaussian graphical model. Biometrika 94 19–35.
  • Zhou, S., Lafferty, J. and Wasserman, L. (2010). Time varying undirected graphs. Mach. Learn. 80 295–329.