The Annals of Statistics

Variable selection in linear mixed effects models

Yingying Fan and Runze Li

Full-text: Open access

Abstract

This paper is concerned with the selection and estimation of fixed and random effects in linear mixed effects models. We propose a class of nonconcave penalized profile likelihood methods for selecting and estimating important fixed effects. To overcome the difficulty of unknown covariance matrix of random effects, we propose to use a proxy matrix in the penalized profile likelihood. We establish conditions on the choice of the proxy matrix and show that the proposed procedure enjoys the model selection consistency where the number of fixed effects is allowed to grow exponentially with the sample size. We further propose a group variable selection strategy to simultaneously select and estimate important random effects, where the unknown covariance matrix of random effects is replaced with a proxy matrix. We prove that, with the proxy matrix appropriately chosen, the proposed procedure can identify all true random effects with asymptotic probability one, where the dimension of random effects vector is allowed to increase exponentially with the sample size. Monte Carlo simulation studies are conducted to examine the finite-sample performance of the proposed procedures. We further illustrate the proposed procedures via a real data example.

Article information

Source
Ann. Statist., Volume 40, Number 4 (2012), 2043-2068.

Dates
First available in Project Euclid: 30 October 2012

Permanent link to this document
https://projecteuclid.org/euclid.aos/1351602536

Digital Object Identifier
doi:10.1214/12-AOS1028

Mathematical Reviews number (MathSciNet)
MR3059076

Zentralblatt MATH identifier
1257.62077

Subjects
Primary: 62J05: Linear regression 62J07: Ridge regression; shrinkage estimators
Secondary: 62F10: Point estimation

Keywords
Adaptive Lasso linear mixed effects models group variable selection oracle property SCAD

Citation

Fan, Yingying; Li, Runze. Variable selection in linear mixed effects models. Ann. Statist. 40 (2012), no. 4, 2043--2068. doi:10.1214/12-AOS1028. https://projecteuclid.org/euclid.aos/1351602536


Export citation

References

  • Akaike, H. (1973). Information theory and an extension of the maximum likelihood principle. In Second International Symposium on Information Theory (Tsahkadsor, 1971) (B. N. Petrov and F. Csaki, eds.) 267–281. Akad. Kiadó, Budapest.
  • Bickel, P. J. and Levina, E. (2004). Some theory of Fisher’s linear discriminant function, “naive Bayes,” and some alternatives when there are many more variables than observations. Bernoulli 10 989–1010.
  • Bondell, H. D., Krishna, A. and Ghosh, S. K. (2010). Joint variable selection for fixed and random effects in linear mixed-effects models. Biometrics 66 1069–1077.
  • Box, G. E. P. and Tiao, G. C. (1973). Bayesian Inference in Statistical Analysis. Addison-Wesley, Reading, MA.
  • Chen, Z. and Dunson, D. B. (2003). Random effects selection in linear mixed models. Biometrics 59 762–769.
  • Dudoit, S., Fridlyand, J. and Speed, T. P. (2002). Comparison of discrimination methods for the classification of tumors using gene expression data. J. Amer. Statist. Assoc. 97 77–87.
  • Efron, B., Hastie, T., Johnstone, I. and Tibshirani, R. (2004). Least angle regression (with discussion). Ann. Statist. 32 407–451.
  • Fan, J. and Fan, Y. (2008). High-dimensional classification using features annealed independence rules. Ann. Statist. 36 2605–2637.
  • Fan, J. and Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. J. Amer. Statist. Assoc. 96 1348–1360.
  • Fan, Y. and Li, R. (2012). Supplement to “Variable selection in linear mixed effects models.” DOI:10.1214/12-AOS1028SUPP.
  • Fan, J. and Peng, H. (2004). Nonconcave penalized likelihood with a diverging number of parameters. Ann. Statist. 32 928–961.
  • Gelman, A., Carlin, J. B., Stern, H. S. and Rubin, D. B. (1995). Bayesian Data Analysis. Chapman & Hall, London.
  • Huang, J. Z., Wu, C. O. and Zhou, L. (2002). Varying-coefficient models and basis function approximations for the analysis of repeated measurements. Biometrika 89 111–128.
  • Ibrahim, J. G., Zhu, H., Garcia, R. I. and Guo, R. (2011). Fixed and random effects selection in mixed effects models. Biometrics 67 495–503.
  • Kaslow, R. A., Ostrow, D. G., Detels, R., Phair, J. P., Polk, B. F. and Rinaldo, C. R. (1987). The multicenter AIDS cohort study: Rationale, organization and selected characteristics of the participants. American Journal Epidemiology 126 310–318.
  • Laird, N. M. and Ware, J. H. (1982). Random-effects models for longitudinal data. Biometrics 38 963–974.
  • Liang, H., Wu, H. and Zou, G. (2008). A note on conditional AIC for linear mixed-effects models. Biometrika 95 773–778.
  • Lin, X. (1997). Variance component testing in generalised linear models with random effects. Biometrika 84 309–326.
  • Liu, Y. and Wu, Y. (2007). Variable selection via a combination of the $L_0$ and $L_1$ penalties. J. Comput. Graph. Statist. 16 782–798.
  • Longford, N. T. (1993). Random Coefficient Models. Oxford Statistical Science Series 11. Oxford Univ. Press, New York.
  • Lv, J. and Fan, Y. (2009). A unified approach to model selection and sparse recovery using regularized least squares. Ann. Statist. 37 3498–3528.
  • Patterson, H. D. and Thompson, R. (1971). Recovery of inter-block information when block sizes are unequal. Biometrika 58 545–554.
  • Pu, W. and Niu, X.-F. (2006). Selecting mixed-effects models based on a generalized information criterion. J. Multivariate Anal. 97 733–758.
  • Qu, A. and Li, R. (2006). Quadratic inference functions for varying-coefficient models with longitudinal data. Biometrics 62 379–391.
  • Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. J. R. Stat. Soc. Ser. B Stat. Methodol. 58 267–288.
  • Vaida, F. and Blanchard, S. (2005). Conditional Akaike information for mixed-effects models. Biometrika 92 351–370.
  • Verbeke, G. and Molenberghs, G. (2000). Linear Mixed Models for Longitudinal Data. Springer, New York.
  • Vrahatis, M. N. (1989). A short proof and a generalization of Miranda’s existence theorem. Proc. Amer. Math. Soc. 107 701–703.
  • Wang, H., Li, R. and Tsai, C.-L. (2007). Tuning parameter selectors for the smoothly clipped absolute deviation method. Biometrika 94 553–568.
  • Yuan, M. and Lin, Y. (2006). Model selection and estimation in regression with grouped variables. J. R. Stat. Soc. Ser. B Stat. Methodol. 68 49–67.
  • Zhang, C.-H. (2010). Nearly unbiased variable selection under minimax concave penalty. Ann. Statist. 38 894–942.
  • Zhang, Y., Li, R. and Tsai, C.-L. (2010). Regularization parameter selections via generalized information criterion. J. Amer. Statist. Assoc. 105 312–323.
  • Zou, H. (2006). The adaptive lasso and its oracle properties. J. Amer. Statist. Assoc. 101 1418–1429.
  • Zou, H. and Hastie, T. (2005). Regularization and variable selection via the elastic net. J. R. Stat. Soc. Ser. B Stat. Methodol. 67 301–320.
  • Zou, H. and Li, R. (2008). One-step sparse estimates in nonconcave penalized likelihood models (with discussion). Ann. Statist. 36 1509–1566.
  • Zou, H. and Zhang, H. H. (2009). On the adaptive elastic-net with a diverging number of parameters. Ann. Statist. 37 1733–1751.

Supplemental materials

  • Supplementary material: Supplement to “Variable selection in linear mixed effects models”. We included additional simulation examples and technical proofs omitted from the main text: simulation Examples A.1–A.3, and technical proofs of Lemmas 1–3 and Proposition 1.