Electronic Journal of Statistics

Generalised linear mixed model analysis via sequential Monte Carlo sampling

Y. Fan, D.S. Leslie, and M.P. Wand

Full-text: Open access


We present a sequential Monte Carlo sampler algorithm for the Bayesian analysis of generalised linear mixed models (GLMMs). These models support a variety of interesting regression-type analyses, but performing inference is often extremely difficult, even when using the Bayesian approach combined with Markov chain Monte Carlo (MCMC). The Sequential Monte Carlo sampler (SMC) is a new and general method for producing samples from posterior distributions. In this article we demonstrate use of the SMC method for performing inference for GLMMs. We demonstrate the effectiveness of the method on both simulated and real data, and find that sequential Monte Carlo is a competitive alternative to the available MCMC techniques.

Article information

Electron. J. Statist., Volume 2 (2008), 916-938.

First available in Project Euclid: 6 October 2008

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

generalised additive models longitudinal data analysis nonparametric regression sequential Monte Carlo sampler


Fan, Y.; Leslie, D.S.; Wand, M.P. Generalised linear mixed model analysis via sequential Monte Carlo sampling. Electron. J. Statist. 2 (2008), 916--938. doi:10.1214/07-EJS158. https://projecteuclid.org/euclid.ejs/1223304591

Export citation


  • [1] Besag, J., Green, P.J., Higdon, D. and Mengersen, K. (1995). Bayesian computation and stochastic systems., Statistical Science, 10, 3–66.
  • [2] Breslow, N.E. and Clayton, D.G. (1993). Approximate inference in generalized linear mixed models., Journal of the American Statistical Association, 88, 9–25.
  • [3] Breslow, N.E. and Lin X. (1995). Bias correction in generalised linear mixed models with a single component of dispersion., Biometrika, 82, 81–91.
  • [4] Carpenter, J., Clifford, P. and Fearnhead, P. (1999). An improved particle filter for non-linear problems., IEE proceedings — Radar, Sonar and Navigation, 146, 2–7.
  • [5] Chopin, N. (2002). A Sequential Particle Filter for Static Models., Biometrika, 89, 539–551.
  • [6] Clayton, D. (1996). Generalized linear mixed models, pp. 275–301. In, Markov Chain Monte Carlo in Practice, eds. Gilks, W.R., Richardson, S. and Spiegelhalter, D. J. London: Chapman & Hall.
  • [7] Cowles, M. K. and Carlin, B. P. (1996). Markov chain Monte Carlo convergence diagnostics: A comparative review., Journal of American Statistics Association, 91, 883–904.
  • [8] Del Moral, P., Doucet, A. and Jasra, A. (2006). Sequential Monte Carlo samplers., Journal of the Royal Statistical Society, Series B, 68, 411–436.
  • [9] Del Moral, P., Doucet, A. and Jasra, A. (2007). Sequential Monte Carlo for Bayesian computation. In, Bayesian Statistics 8, eds. J. M. Bernardo, M. J. Bayarri, J. O. Berger, A. P. Dawid, D. Heckerman, A. F. M. Smith and M. West.
  • [10] Diggle, P., Liang, K-L. and Zeger, S. (1995)., Analysis of Longitudinal Data. Oxford: Oxford University Press.
  • [11] Douc, R., Cappé, O. and Moulines, E. (2005). Comparison of resampling schemes for particle filtering., Proceedings of the 4th International Symposium on Image and Signal Processing and Analysis, 64–69.
  • [12] Doucet, A., Godsill, S. and Andrieu C. (2000). On sequential Monte Carlo sampling methods for Bayesian filtering., Statistics and Computing, 10, 197–208.
  • [13] Fearnhead, P. (2004). Particle filters for mixture models with an unknown number of components., Statistics and Computing, 14, 11–21.
  • [14] Frigessi, A. (2003). On Some Current Research in MCMC, pp. 172–178. In, Highly Structured Stochastic Systems, eds. Green, P.J., Hjort, N.L. and Richardson, S. Oxford University Press.
  • [15] Gamerman, D. (1997)., Markov chain Monte Carlo: Stochastic simulation for Bayesian inference. Chapman & Hall.
  • [16] Gelfand, A.E., Sahu, S.K. and Carlin, B.P. (1995). Efficient parameterisations for normal linear mixed models., Biometrika, 82, 479–488.
  • [17] Gelfand, A.E. and Smith, A. F. M. (1990). Sampling Based Approaches to Calculating Marginal Densities., Journal of the American Statistical Association, 85, 398–409.
  • [18] Gelman, A. (2006). Prior distributions for variance parameters in hierarchical models., Bayesian Analysis, 1, 515–533.
  • [19] Gilks, W. R. and Berzuini, C. (2001). Following a moving target—Monte Carlo inference for dynamic Bayesian models., Journal of the Royal Statistical Society, Series B, 63, 127–146.
  • [20] Kong, A., Liu, J.S., and Wong, W.H. (1994). Sequential imputations and Bayesian missing data problems., Journal of American Statistics Association, 89(425), 278–288.
  • [21] Kitagawa, G. (1996). Monte Carlo filter and smoother for non-Gaussian, non-linear state space models., Journal of Computational and Graphical Statistics, 5, 1–25.
  • [22] Lin, X. and Carroll, R.J. (2001) Semiparametric regression for clustered data., Biometrika, 88, 1179–1865.
  • [23] McCulloch, C.E., and Searle, S.R. (2000)., Generalized, Linear, and Mixed Models. New York: John Wiley & Sons.
  • [24] Neal, R. (2001). Annealed importance sampling., Statistics and Computing, 11, 125–139.
  • [25] Roberts, G. O., Gelman, A. and Gilks, W. R. (1997). Weak convergence and optimal scaling of random walk Metropolis algorithms., Annals of Applied Probability, 7, 110–120.
  • [26] Roberts, G. O. and Sahu, S. K. (1997). Updating schemes, covariance structure, blocking and parameterisation for the Gibbs sampler., Journal of the Royal Statistical Society, Series B, 59, 291–317.
  • [27] Ruppert, D., Wand, M. P. and Carroll, R.J. (2003)., Semiparametric Regression. New York: Cambridge University Press.
  • [28] Skrondal, A. and Rabe-Hesketh, S. (2004)., Generalized Latent Variable Modeling: Multilevel, Longitudinal and Structural Equation Models. Boca Raton, Florida: Chapman & Hall.
  • [29] Venables, W. & Ripley, B.D. (2005). MASS. R package within the bundle VR 7.2-24., http://cran.r-project.org.
  • [30] Zhao, Y., Staudenmayer, J., Coull, B.A. and Wand, M.P. (2006). General design Bayesian generalized linear mixed models., Statistical Science, 21, 35–51.