The Annals of Statistics

Asymptotic normality and valid inference for Gaussian variational approximation

Peter Hall, Tung Pham, M. P. Wand, and S. S. J. Wang

Full-text: Open access


We derive the precise asymptotic distributional behavior of Gaussian variational approximate estimators of the parameters in a single-predictor Poisson mixed model. These results are the deepest yet obtained concerning the statistical properties of a variational approximation method. Moreover, they give rise to asymptotically valid statistical inference. A simulation study demonstrates that Gaussian variational approximate confidence intervals possess good to excellent coverage properties, and have a similar precision to their exact likelihood counterparts.

Article information

Ann. Statist. Volume 39, Number 5 (2011), 2502-2532.

First available in Project Euclid: 30 November 2011

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62F12: Asymptotic properties of estimators
Secondary: 62F25: Tolerance and confidence regions

Generalized linear mixed models longitudinal data analysis maximum likelihood estimation Poisson mixed models


Hall, Peter; Pham, Tung; Wand, M. P.; Wang, S. S. J. Asymptotic normality and valid inference for Gaussian variational approximation. Ann. Statist. 39 (2011), no. 5, 2502--2532. doi:10.1214/11-AOS908.

Export citation


  • [1] Bates, D. and Maechler, M. (2010). lme4: Linear mixed-effects models using S4 classes. R package. Available at
  • [2] Beaumont, M. A., Zhang, W. and Balding, D. J. (2002). Approximate Bayesian computation in population genetics. Genetics 162 2025–2035.
  • [3] Bishop, C. M. (2006). Pattern Recognition and Machine Learning. Springer, New York.
  • [4] Boccignone, G., Napoletano, P. and Ferraro, M. (2008). Embedding diffusion in variational Bayes: A technique for segmenting images. International Journal of Pattern Recognition and Artificial Intelligence 22 811–827.
  • [5] Del Moral, P., Doucet, A. and Jasra, A. (2006). Sequential Monte Carlo samplers. J. R. Stat. Soc. Ser. B Stat. Methodol. 68 411–436.
  • [6] Flandin, G. and Penny, W. D. (2007). Bayesian fMRI data analysis with sparse spatial basis function priors. NeuroImage 45 S173–S186.
  • [7] Hall, P., Ormerod, J. T. and Wand, M. P. (2011). Theory of Gaussian variational approximation for a Poisson mixed model. Statist. Sinica 21 369–389.
  • [8] Logsdon, B. A., Hoffman, G. E. and Mezey, J. G. (2010). A variational Bayes algorithm for fast and accurate multiple locus genome-wide association analysis. BMC Bioinformatics 11 1–13.
  • [9] McCulloch, C. E., Searle, S. R. and Neuhaus, J. M. (2008). Generalized, Linear, and Mixed Models, 2nd ed. Wiley, Hoboken, NJ.
  • [10] McGrory, C. A., Titterington, D. M., Reeves, R. and Pettitt, A. N. (2009). Variational Bayes for estimating the parameters of a hidden Potts model. Stat. Comput. 19 329–340.
  • [11] Minka, T. (2001). Expectation propagation for approximate Bayesian inference. In Proceedings of Conference on Uncertainty in Artificial Intelligence 362–369. Univ. Washington, Seattle.
  • [12] Minka, T., Winn, J., Guiver, J. and Kannan, A. (2010). Infer.Net 2.4, Microsoft Research Cambridge, Cambridge, UK.
  • [13] Ormerod, J. T. and Wand, M. P. (2010). Explaining variational approximations. Amer. Statist. 64 140–153.
  • [14] Ormerod, J. T. and Wand, M. P. (2011). Gaussian variational approximate inference for generalized linear mixed models. J. Comput. Graph. Statist. 20. To appear. DOI:10.1198/jcgs.2011.09118.
  • [15] R Development Core Team. (2010). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. Available at
  • [16] Rue, H., Martino, S. and Chopin, N. (2009). Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations (with discussion). J. R. Stat. Soc. Ser. B Stat. Methodol. 71 319–392.
  • [17] Teschendorff, A. E., Wang, Y., Barbosa-Morais, N. L., Brenton, J. D. and Caldas, C. (2005). A variational Bayesian mixture modelling framework for cluster analysis of gene-expression data. Bioinformatics 21 3025–3033.