Electronic Journal of Statistics

Approximately exact calculations for linear mixed models

Michael Lavine, Andrew Bray, and Jim Hodges

Full-text: Open access


This paper is about computations for linear mixed models having two variances, $\sigma^{2}_{e}$ for residuals and $\sigma^{2}_{s}$ for random effects, though the ideas can be extended to some linear mixed models having more variances. Researchers are often interested in either the restricted (residual) likelihood $\text{RL}(\sigma_{e}^{2},\sigma_{s}^{2})$ or the joint posterior $\pi(\sigma_{e}^{2},\sigma_{s}^{2}\,|\,y)$ or their logarithms. Both $\log\text{RL}$ and $\log\pi$ can be multimodal and computations often rely on either a general purpose optimization algorithm or MCMC, both of which can fail to find regions where the target function is high. This paper presents an alternative. Letting $f$ stand for either $\text{RL}$ or $\pi$, we show how to find a box $B$ in the $(\sigma_{e}^{2},\sigma_{s}^{2})$ plane such that

1. all local and global maxima of $\log f$ lie within $B$;

2. $\sup_{(\sigma_{e}^{2},\sigma_{s}^{2})\in B^{c}}\log f(\sigma_{e}^{2},\sigma_{s}^{2})\le \sup_{(\sigma_{e}^{2},\sigma_{s}^{2})\in B}\log f(\sigma_{e}^{2},\sigma_{s}^{2})-M$ for a prespecified $M>0$; and

3. $\log f$ can be estimated to within a prespecified tolerance $\epsilon$ everywhere in $B$ with no danger of missing regions where $\log f$ is large.

Taken together these conditions imply that the $(\sigma_{e}^{2},\sigma_{s}^{2})$ plane can be divided into two parts: $B$, where we know $\log f$ as accurately as we wish, and $B^{c}$, where $\log f$ is small enough to be safely ignored. We provide algorithms to find $B$ and to evaluate $\log f$ as accurately as desired everywhere in $B$.

Article information

Electron. J. Statist., Volume 9, Number 2 (2015), 2293-2323.

Received: January 2015
First available in Project Euclid: 13 October 2015

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62J05: Linear regression
Secondary: 62F99: None of the above, but in this section


Lavine, Michael; Bray, Andrew; Hodges, Jim. Approximately exact calculations for linear mixed models. Electron. J. Statist. 9 (2015), no. 2, 2293--2323. doi:10.1214/15-EJS1072. https://projecteuclid.org/euclid.ejs/1444742888

Export citation


  • [1] Browne, W., Goldstein, H., and Rasbash, J. (2001), “Multiple Membership Multiple Classification (MMMC) Models,”, Statistical Modeling, 1, 103–124.
  • [2] Bryk, A. S. and Raudenbush, S. W. (1992), Hierarchical Linear Models: Applications and Data Analysis Methods, Sage, Newbury Park.
  • [3] Henn, L. and Hodges, J. S. (2014), “Multiple Local Maxima in Restricted Likelihoods and Posterior Distributions for Mixed Linear Models,”, International Statistical Review, 82, 90–105.
  • [4] Hill, B. (1965), “Inference About Variance Components in the One-way Model,”, Journal of the American Statistical Association, 60, 806–825.
  • [5] Hodges, J. H. (1998), “Some Algebra and Geometry for Hierarchical Models Applied to Diagnostics,”, JRSS B, 60, 497–536.
  • [6] Hodges, J. S. (2013), Richly Parameterized Linear Models: Additive, Time Series, and Spatial Models Using Random Effects, CRC Press.
  • [7] Houtman, A. and Speed, T. (1983), “Balance in Designed Experiments with Orthogonal Block Structure,”, Annals of Statistics, 11, 1069–1085.
  • [8] Liu, J. and Hodges, J. S. (2003), “Posterior Bimodality in the Balanced One-way Random-effects Model,”, Journal of the Royal Statistical Society: Series B (Statistical Methodology), 65, 247–255.
  • [9] McCaffrey, D., Lockwood, J., Koretz, D., Louis, T., and Hamilton, L. (2004), “Models for Value-added Modeling of Teacher Effects,”, Journal of Behavioral and Educational Statistics, 29, 67–101.
  • [10] Mullen, K. M. (2014), “Continuous Global Optimization in R,”, Journal of Statistical Software, 60.
  • [11] Reich, B. and Hodges, J. (2008), “Identification of the Variance Components in the General Two-variance Linear Model,”, JSPI, 138, 1592–1604.
  • [12] Reiss, P. T., Huang, L., Chen, Y.-H., Huo, L., Tarpey, T., and Mennes, M. (2014), “Massively Parallel Nonparametric Regression, with an Application to Developmental Brain Mapping,”, Journal of Computational and Graphical Statistics, 23, 232–248.
  • [13] Ruppert, D., Wand, M. P., and Carroll, R. J. (2003), Semiparametric Regression, Cambridge University Press, Cambridge.
  • [14] Verbeke, G. and Molenberghs, G. (2000), Linear Mixed Models for Longitudinal Data, Springer, first edn.
  • [15] Wakefield, J. (1998), “Comment on Some Algebra and Geometry for Hierarchical Models Applied to Diagnostics,”, Journal of the Royal Statistical Society (Series B), 60, 497–536.
  • [16] Welham, S. and Thompson, R. (2009), “A Note on Bimodality in the Log-likelihood Function for Penalized Spline Mixed Models,”, Computational Statistics and Data Analysis, 53, 920–931.
  • [17] West, B. T., Welch, Kathleen, B., and Galecki, A. T. (2014), Linear Mixed Models: A Practical Guide Using Statistical Software, CRC Press, second edn.