Bayesian Analysis

Efficient Model Comparison Techniques for Models Requiring Large Scale Data Augmentation

Panayiota Touloupou, Naif Alzahrani, Peter Neal, Simon E. F. Spencer, and Trevelyan J. McKinley

Full-text: Open access

Enhanced PDF (499 KB)

Abstract

Selecting between competing statistical models is a challenging problem especially when the competing models are non-nested. In this paper we offer a simple solution by devising an algorithm which combines MCMC and importance sampling to obtain computationally efficient estimates of the marginal likelihood which can then be used to compare the models. The algorithm is successfully applied to a longitudinal epidemic data set, where calculating the marginal likelihood is made more challenging by the presence of large amounts of missing data. In this context, our importance sampling approach is shown to outperform existing methods for computing the marginal likelihood.

Article information

Source
Bayesian Anal., Volume 13, Number 2 (2018), 437-459.

Dates
First available in Project Euclid: 29 April 2017

Permanent link to this document
https://projecteuclid.org/euclid.ba/1493431262

Digital Object Identifier
doi:10.1214/17-BA1057

Mathematical Reviews number (MathSciNet)
MR3780430

Zentralblatt MATH identifier
06989955

Keywords
epidemics marginal likelihood model evidence model selection

Rights
Creative Commons Attribution 4.0 International License.

Citation

Touloupou, Panayiota; Alzahrani, Naif; Neal, Peter; Spencer, Simon E. F.; McKinley, Trevelyan J. Efficient Model Comparison Techniques for Models Requiring Large Scale Data Augmentation. Bayesian Anal. 13 (2018), no. 2, 437--459. doi:10.1214/17-BA1057. https://projecteuclid.org/euclid.ba/1493431262


Export citation

References

  • Addy, C. L., Longini, I. M. and Haber, M. (1991). A generalized stochastic model for the analysis of infectious disease final size data. Biometrics 47, 961–974.
    Zentralblatt MATH: 0729.62564
    Digital Object Identifier: doi:10.2307/2532652
  • Anderson, B. D. O. and Moore, J. B. (1979). Optimal filtering. Englewood Cliffs, New Jersey: Prentice Hall.
    Zentralblatt MATH: 0688.93058
  • Auranen K., Arjas E., Leino T. and Takala A. K. (2000). Transmission of pneummonococcal carriage in families: A latent Markov process model for binary longitudinal data. Journal of the American Statistical Association 95, 1044–1053.
    Zentralblatt MATH: 1008.62106
    Digital Object Identifier: doi:10.1080/01621459.2000.10474301
  • Ball, F. G., Mollison, D. and Scalia-Tomba, G. (1997). Epidemics with two levels of mixing. The Annals of Applied Probability 7, 46–89.
    Zentralblatt MATH: 0909.92028
    Digital Object Identifier: doi:10.1214/aoap/1034625252
    Project Euclid: euclid.aoap/1034625252
  • Carter, C. K. and Kohn, R. (1994). On Gibbs sampling for state space models. Biometrika 81, 541–553.
    Zentralblatt MATH: 0809.62087
    Digital Object Identifier: doi:10.1093/biomet/81.3.541
  • Chen, M.-H. (2005). Computing marginal likelihoods from a single MCMC output. Statistica Neerlandica 59, 16–29.
    Zentralblatt MATH: 1069.62023
    Digital Object Identifier: doi:10.1111/j.1467-9574.2005.00276.x
  • Chib, S. (1995). Marginal likelihood from the Gibbs output. Journal of the American Statistical Association 90, 1313–1321.
    Zentralblatt MATH: 0868.62027
    Digital Object Identifier: doi:10.1080/01621459.1995.10476635
  • Chib, S. and Jeliazkov, I. (2001). Marginal likelihood from the Metropolis-Hastings output. Journal of the American Statistical Association 96, 270–281.
    Zentralblatt MATH: 1015.62020
    Digital Object Identifier: doi:10.1198/016214501750332848
  • Clancy, D. and O’Neill, P. (2007). Exact Bayesian inference and model selection for stochastic models of epidemics among a community of households. Scandinavian Journal of Statistics 34, 259–274.
    Zentralblatt MATH: 1142.62099
    Digital Object Identifier: doi:10.1111/j.1467-9469.2006.00522.x
  • Clyde, M. A., Berger, J. O., Bullard, F., Ford, E. B., Jefferys, W. H., Luo, R., Paulo, R. and Loredo, T. (2007). Current challenges in Bayesian model choice. In Statistical Challenges in Modern Astronomy IV ASP Conference Series, Vol. 371, proceedings of the conference held 12–15 June 2006 at Pennsylvania State University, in University Park, Pennsylvania, USA. Edited by G. Jogesh Babu and Eric D. Feigelson. p. 224.
  • Doucet, A. and Johansen, A. M. (2011). A tutorial on particle filtering and smoothing: fifteen years later. In Handbook of Nonlinear Filtering (eds D. Crisan and B. Rozovsky). Cambridge: Cambridge University Press. pp. 656–704.
    Mathematical Reviews (MathSciNet): MR2884612
    Zentralblatt MATH: 05919872
  • Fong, Y., Rue, H. and Wakefield, J. (2010). Bayesian inference for generalized linear mixed models. Biostatistics 11, 397–412.
  • Fox, J. P. and Hall, C. E. (1980). Viruses in families. PSG Publishing, Littleton, MA.
  • Friel, N. and Pettitt, A. N. (2008). Marginal likelihood estimation via power posteriors. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 70, 589–607.
    Zentralblatt MATH: 05563360
    Digital Object Identifier: doi:10.1111/j.1467-9868.2007.00650.x
  • Gelfand, A. E. and Dey, D. K. (1994). Bayesian model choice. Exact and asymptotic calculations. Journal of the Royal Statistical Society. Series B (Methodological) 56, 501–514.
    Zentralblatt MATH: 0800.62170
  • Gordon, N. J., Salmond, D. J., and Smith, A. F. M. (1993). Novel approach to nonlinear/non-Gaussian Bayesian state estimation. Radar and Signal Processing, IEE Proceedings F 140, 107–113.
  • Green P.J. (1995). Reversible jump Markov chain Monte Carlo computation and Bayesian model determination. Biometrika 82, 711–732.
    Zentralblatt MATH: 0861.62023
    Digital Object Identifier: doi:10.1093/biomet/82.4.711
  • Hastings, W. (1970). Monte Carlo sampling methods using Markov chains and their applications. Biometrika 57, 97–109.
    Zentralblatt MATH: 0219.65008
    Digital Object Identifier: doi:10.1093/biomet/57.1.97
  • Hesterberg, T. (1995). Weighted average importance sampling and defense mixture distributions. Technometrics 37, 185–194.
    Zentralblatt MATH: 0822.62002
    Digital Object Identifier: doi:10.1080/00401706.1995.10484303
  • Hussain M., Melegaro A., Pebody R. G., George R., Edmunds W. J., Talukdar R., Martin S. A., Efstratiou A. and Miller E. (2005). A longitudinal household study of Streptococcus pneumoniae nasopharyngeal carriage in a UK setting. Epidemiology and Infection 5, 891–898.
  • Karagiannis, G., and Andrieu, C. (2013). Annealed importance sampling reversible jump MCMC algorithms. Journal of Computational and Graphical Statistics 22, 623–648.
  • Kass, R. E. and Raftery, A. E. (1995). Bayes factors. Journal of the American Statistical Association 90, 773–795.
    Zentralblatt MATH: 0846.62028
    Digital Object Identifier: doi:10.1080/01621459.1995.10476572
  • Kohn, R., Smith, M., Chan D. (2001). Nonparametric regression using linear combinations of basis functions. Statistics and Computing 11, 313–322.
  • Melegaro, A., Gay, N., and Medley, G. (2004). Estimating the transmission parameters of pneumococcal carriage in households. Epidemiology and Infection 132, 433–441.
  • Metropolis N., Rosenbluth A., Rosenbluth M., Teller A. and Teller E. (1953). Equations of state calculations by fast computing machines. Journal of Chemical Physics 21, 1087–1092.
  • Meng, X.-L. and Wong, W. H. (1996). Simulating ratios of normalizing constants via a simple identity: A theoretical exploration. Statistica Sinica 6, 831–860.
    Zentralblatt MATH: 0857.62017
  • Monto, A. S., Koopman, J. S. and Longini, I. M. (1985). Tecumseh study of illness. XIII. Influenza infection and disease, 1976–1981. American Journal of Epidemiology 121, 811–822.
  • Neal, P. and Kypraios, T. (2015). Exact Bayesian inference via data augmentation. Statistics and Computing 25, 333–347
    Zentralblatt MATH: 1331.62035
    Digital Object Identifier: doi:10.1007/s11222-013-9435-z
  • Neal, P. and Subba Rao, T. (2007). MCMC for integer-valued ARMA processes. Journal of Time Series Analysis 28, 92–110.
    Zentralblatt MATH: 1164.62049
    Digital Object Identifier: doi:10.1111/j.1467-9892.2006.00500.x
  • Newton, M. A. and Raftery, A. E. (1994). Approximate Bayesian inference with the weighted likelihood bootstrap. Journal of the Royal Statistical Society. Series B (Methodological) 56, 3–48.
    Zentralblatt MATH: 0788.62026
  • O’Neill, P. D., Balding, D. J., Becker, N. G., Eerola, M. and Mollison, D. (2000). Analyses of infectious disease data from household outbreaks by Markov Chain Monte Carlo methods. Journal of the Royal Statistical Society. Series C (Applied Statistics) 49, 517–542.
    Mathematical Reviews (MathSciNet): MR1824557
    Zentralblatt MATH: 0965.62098
    Digital Object Identifier: doi:10.1111/1467-9876.00210
  • Ripley, B. D. (1987). Stochastic Simulation. Wiley & Sons.
    Zentralblatt MATH: 0613.65006
  • Roberts, G. O. and Rosenthal, J. S. (2009). Examples of adaptive MCMC. Journal of Computational and Graphical Statistics 18, 349–367
  • Skilling, J. (2004). Nested sampling. AIP Conference Proceedings 735, 395–405.
  • Smith, M. and Kohn, R. (1996). Nonparametric regression using Bayesian variable selection. Journal of Econometrics 75, 317–344.
    Zentralblatt MATH: 0864.62025
    Digital Object Identifier: doi:10.1016/0304-4076(95)01763-1
  • Tierney, L. and Kadane, J. B. (1986). Accurate approximations for posterior moments and marginal densities. Journal of the American Statistical Association 81, 82–86. 590–599.
    Zentralblatt MATH: 0587.62067
    Digital Object Identifier: doi:10.1080/01621459.1986.10478240
  • Touloupou, P., Alzahrani, N., Neal, P., Spencer, S. E. F., and McKinley, T. J. (2017). Supplementary material: Efficient model comparison techniques for models requiring large scale data augmentation. Bayesian Analysis.
  • Zeger, S. (1988). A regression model for time series of counts. Biometrika 75, 621–629.
    Zentralblatt MATH: 0653.62064
    Digital Object Identifier: doi:10.1093/biomet/75.4.621
  • Zhou, Y., Johansen, A. M. and Aston, J. A. D. (2015). Towards automatic model comparison: An adaptive sequential Monte Carlo approach. Journal of Computational and Graphical Statistics 25, 701–726.

Supplemental materials

International Society for Bayesian Analysis

Bayesian Analysis

New content alerts

Email RSS ToC RSS Article
  • You have access to this content.
  • You have partial access to this content.
  • You do not have access to this content.
Turn Off MathJax

What is MathJax?

More like this

+ See more