The Annals of Statistics

Regularized rank-based estimation of high-dimensional nonparanormal graphical models

Lingzhou Xue and Hui Zou

Full-text: Open access

Abstract

A sparse precision matrix can be directly translated into a sparse Gaussian graphical model under the assumption that the data follow a joint normal distribution. This neat property makes high-dimensional precision matrix estimation very appealing in many applications. However, in practice we often face nonnormal data, and variable transformation is often used to achieve normality. In this paper we consider the nonparanormal model that assumes that the variables follow a joint normal distribution after a set of unknown monotone transformations. The nonparanormal model is much more flexible than the normal model while retaining the good interpretability of the latter in that each zero entry in the sparse precision matrix of the nonparanormal model corresponds to a pair of conditionally independent variables. In this paper we show that the nonparanormal graphical model can be efficiently estimated by using a rank-based estimation scheme which does not require estimating these unknown transformation functions. In particular, we study the rank-based graphical lasso, the rank-based neighborhood Dantzig selector and the rank-based CLIME. We establish their theoretical properties in the setting where the dimension is nearly exponentially large relative to the sample size. It is shown that the proposed rank-based estimators work as well as their oracle counterparts defined with the oracle data. Furthermore, the theory motivates us to consider the adaptive version of the rank-based neighborhood Dantzig selector and the rank-based CLIME that are shown to enjoy graphical model selection consistency without assuming the irrepresentable condition for the oracle and rank-based graphical lasso. Simulated and real data are used to demonstrate the finite performance of the rank-based estimators.

Article information

Source
Ann. Statist., Volume 40, Number 5 (2012), 2541-2571.

Dates
First available in Project Euclid: 4 February 2013

Permanent link to this document
https://projecteuclid.org/euclid.aos/1359987530

Digital Object Identifier
doi:10.1214/12-AOS1041

Mathematical Reviews number (MathSciNet)
MR3097612

Zentralblatt MATH identifier
1373.62138

Subjects
Primary: 62G05: Estimation 62G20: Asymptotic properties
Secondary: 62F12: Asymptotic properties of estimators 62J07: Ridge regression; shrinkage estimators

Keywords
CLIME Dantzig selector graphical lasso nonparanormal graphical model rate of convergence variable transformation

Citation

Xue, Lingzhou; Zou, Hui. Regularized rank-based estimation of high-dimensional nonparanormal graphical models. Ann. Statist. 40 (2012), no. 5, 2541--2571. doi:10.1214/12-AOS1041. https://projecteuclid.org/euclid.aos/1359987530


Export citation

References

  • Banerjee, O., El Ghaoui, L. and d’Aspremont, A. (2008). Model selection through sparse maximum likelihood estimation for multivariate Gaussian or binary data. J. Mach. Learn. Res. 9 485–516.
  • Bertsimas, D. and Tsitsiklis, J. N. (1997). Introduction to Linear Optimization. Athena Scientific, Belmont, MA.
  • Bickel, P. J., Ritov, Y. and Tsybakov, A. B. (2009). Simultaneous analysis of lasso and Dantzig selector. Ann. Statist. 37 1705–1732.
  • Boyd, S. and Vandenberghe, L. (2004). Convex Optimization. Cambridge Univ. Press, Cambridge.
  • Cai, T., Liu, W. and Luo, X. (2011). A constrained $\ell_1$ minimization approach to sparse precision matrix estimation. J. Amer. Statist. Assoc. 106 594–607.
  • Candes, E. and Tao, T. (2007). The Dantzig selector: Statistical estimation when $p$ is much larger than $n$. Ann. Statist. 35 2313–2351.
  • Chen, X. and Fan, Y. (2006). Estimation of copula-based semiparametric time series models. J. Econometrics 130 307–335.
  • Chen, X., Fan, Y. and Tsyrennikov, V. (2006). Efficient estimation of semiparametric multivariate copula models. J. Amer. Statist. Assoc. 101 1228–1240.
  • Dempster, A. (1972). Covariance selection. Biometrics 28 157–175.
  • Devlin, S. J., Gnanadesikan, R. and Kettenring, J. R. (1975). Robust estimation and outlier detection with correlation coefficients. Biometrika 62 531–545.
  • Dicker, L. and Lin, X. (2009). Variable selection using the Dantzig selector: Asymptotic theory and extensions. Unpublished manuscript.
  • Dobra, A., Eicher, T. S. and Lenkoski, A. (2010). Modeling uncertainty in macroeconomic growth determinants using Gaussian graphical models. Stat. Methodol. 7 292–306.
  • Drton, M. and Perlman, M. D. (2004). Model selection for Gaussian concentration graphs. Biometrika 91 591–602.
  • Drton, M. and Perlman, M. D. (2007). Multiple testing and error control in Gaussian graphical model selection. Statist. Sci. 22 430–449.
  • Edwards, D. (2000). Introduction to Graphical Modelling, 2nd ed. Springer, New York.
  • Fan, J. and Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. J. Amer. Statist. Assoc. 96 1348–1360.
  • Friedman, N. (2004). Inferring cellular networks using probabilistic graphical models. Science 303 799–805.
  • Friedman, J., Hastie, T. and Tibshirani, R. (2008). Sparse inverse covariance estimation with the graphical lasso. Biostatistics 9 432–441.
  • Hoeffding, W. (1948). A class of statistics with asymptotically normal distribution. Ann. Math. Statistics 19 293–325.
  • James, G. M., Radchenko, P. and Lv, J. (2009). DASSO: Connections between the Dantzig selector and lasso. J. R. Stat. Soc. Ser. B Stat. Methodol. 71 127–142.
  • Kendall, M. G. (1948). Rank Correlation Methods. Charles Griffin and Co. Ltd., London.
  • Kruskal, W. H. (1958). Ordinal measures of association. J. Amer. Statist. Assoc. 53 814–861.
  • Lam, C. and Fan, J. (2009). Sparsistency and rates of convergence in large covariance matrix estimation. Ann. Statist. 37 4254–4278.
  • Laule, O., Fürholz, A., Chang, H. S., Zhu, T., Wang, X., Heifetz, P. B., Gruissem, W. and Lange, M. (2003). Crosstalk between cytosolic and plastidial pathways of isoprenoid biosynthesis in Arabidopsis thaliana. Proc. Natl. Acad. Sci. USA 100 6866–6871.
  • Lauritzen, S. L. (1996). Graphical Models. Oxford Statistical Science Series 17. The Clarendon Press Oxford Univ. Press, New York.
  • Lehmann, E. L. (1998). Nonparametrics: Statistical Methods Based on Ranks. Prentice Hall Upper Saddle River, New Jersey.
  • Li, H. and Gui, J. (2006). Gradient directed regularization for sparse Gaussian concentration graphs, with applications to inference of genetic networks. Biostatistics 7 302–317.
  • Liu, H., Lafferty, J. and Wasserman, L. (2009). The nonparanormal: Semiparametric estimation of high dimensional undirected graphs. J. Mach. Learn. Res. 10 2295–2328.
  • Liu, H., Han, F., Yuan, M., Lafferty, J. and Wasserman, L. (2012). High dimensional semiparametric Gaussian copula graphical models. Technical report, Johns Hopkins Univ.
  • McDiarmid, C. (1989). On the method of bounded differences. In Surveys in Combinatorics, 1989 (Norwich, 1989). London Mathematical Society Lecture Note Series 141 148–188. Cambridge Univ. Press, Cambridge.
  • Meinshausen, N. and Bühlmann, P. (2006). High-dimensional graphs and variable selection with the lasso. Ann. Statist. 34 1436–1462.
  • Peng, J., Wang, P., Zhou, N. and Zhu, J. (2009). Partial correlation estimation by joint sparse regression models. J. Amer. Statist. Assoc. 104 735–746.
  • Ravikumar, P., Wainwright, M. J., Raskutti, G. and Yu, B. (2011). High-dimensional covariance estimation by minimizing $\ell_1$-penalized log-determinant divergence. Electron. J. Statist. 5 935–980.
  • Rodríguez-Concepción, M., Forés, O., Martinez-García, J. F., González, V., Phillips, M. A., Ferrer, A. and Boronat, A. (2004). Distinct light-mediated pathways regulate the biosynthesis and exchange of isoprenoid precursors during Arabidopsis seedling development. Plant Cell 16 144–156.
  • Rothman, A. J., Bickel, P. J., Levina, E. and Zhu, J. (2008). Sparse permutation invariant covariance estimation. Electron. J. Stat. 2 494–515.
  • Song, P. X.-K. (2000). Multivariate dispersion models generated from Gaussian copula. Scand. J. Stat. 27 305–320.
  • Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. J. Roy. Statist. Soc. Ser. B 58 267–288.
  • van de Geer, S., Bühlmann, P. and Zhou, S. (2011). The adaptive and the thresholded Lasso for potentially misspecified models (and a lower bound for the Lasso). Electron. J. Stat. 5 688–749.
  • Wille, A., Zimmermann, P., Vranová, E., Fürholz, A., Laule, O., Bleuler, S., Hennig, L., Prelic, A., Von Rohr, P., Thiele, L. et al. (2004). Sparse graphical Gaussian modeling of the isoprenoid gene network in Arabidopsis thaliana. Genome Biology 5 1–13.
  • Xue, L. and Zou, H. (2011a). On estimating sparse correlation matrices of semiparametric Gaussian copulas. Technical report, Univ. Minnesota.
  • Xue, L. and Zou, H. (2011b). Regularized rank-based estimation of high-dimensional nonparanormal graphical models. Technical report, Univ. Minnesota.
  • Xue, L. and Zou, H. (2012). Supplement to “Regularized rank-based estimation of high-dimensional nonparanormal graphical models.” DOI:10.1214/12-AOS1041SUPP.
  • Yuan, M. (2010). High dimensional inverse covariance matrix estimation via linear programming. J. Mach. Learn. Res. 11 2261–2286.
  • Yuan, M. and Lin, Y. (2007). Model selection and estimation in the Gaussian graphical model. Biometrika 94 19–35.
  • Zhou, S., Rütimann, P., Xu, M. and Bühlmann, P. (2011). High-dimensional covariance estimation based on Gaussian graphical models. J. Mach. Learn. Res. 12 2975–3026.
  • Zou, H. (2006). The adaptive lasso and its oracle properties. J. Amer. Statist. Assoc. 101 1418–1429.

Supplemental materials

  • Supplementary material: Supplement material for “Regularized rank-based estimation of high-dimensional nonparanormal graphical models”. In this supplementary note, we give the complete proofs of Theorems 2 and 5.