## Statistical Science

### Bayesian Indirect Inference Using a Parametric Auxiliary Model

#### Abstract

Indirect inference (II) is a methodology for estimating the parameters of an intractable (generative) model on the basis of an alternative parametric (auxiliary) model that is both analytically and computationally easier to deal with. Such an approach has been well explored in the classical literature but has received substantially less attention in the Bayesian paradigm. The purpose of this paper is to compare and contrast a collection of what we call parametric Bayesian indirect inference (pBII) methods. One class of pBII methods uses approximate Bayesian computation (referred to here as ABC II) where the summary statistic is formed on the basis of the auxiliary model, using ideas from II. Another approach proposed in the literature, referred to here as parametric Bayesian indirect likelihood (pBIL), uses the auxiliary likelihood as a replacement to the intractable likelihood. We show that pBIL is a fundamentally different approach to ABC II. We devise new theoretical results for pBIL to give extra insights into its behaviour and also its differences with ABC II. Furthermore, we examine in more detail the assumptions required to use each pBII method. The results, insights and comparisons developed in this paper are illustrated on simple examples and two other substantive applications. The first of the substantive examples involves performing inference for complex quantile distributions based on simulated data while the second is for estimating the parameters of a trivariate stochastic process describing the evolution of macroparasites within a host based on real data. We create a novel framework called Bayesian indirect likelihood (BIL) that encompasses pBII as well as general ABC methods so that the connections between the methods can be established.

#### Article information

Source
Statist. Sci. Volume 30, Number 1 (2015), 72-95.

Dates
First available in Project Euclid: 4 March 2015

http://projecteuclid.org/euclid.ss/1425492441

Digital Object Identifier
doi:10.1214/14-STS498

Mathematical Reviews number (MathSciNet)
MR3317755

Zentralblatt MATH identifier
1332.62088

#### Citation

Drovandi, Christopher C.; Pettitt, Anthony N.; Lee, Anthony. Bayesian Indirect Inference Using a Parametric Auxiliary Model. Statist. Sci. 30 (2015), no. 1, 72--95. doi:10.1214/14-STS498. http://projecteuclid.org/euclid.ss/1425492441.

#### References

• Andrieu, C. and Roberts, G. O. (2009). The pseudo-marginal approach for efficient Monte Carlo computations. Ann. Statist. 37 697–725.
• Beaumont, M. A., Zhang, W. and Balding, D. J. (2002). Approximate Bayesian computation in population genetics. Genetics 162 2025–2035.
• Billingsley, P. (1999). Convergence of Probability Measures, 2nd ed. Wiley, New York.
• Blum, M. G. B. (2010). Approximate Bayesian computation: A nonparametric perspective. J. Amer. Statist. Assoc. 105 1178–1187.
• Blum, M. G. B., Nunes, M. A., Prangle, D. and Sisson, S. A. (2013). A comparative review of dimension reduction methods in approximate Bayesian computation. Statist. Sci. 28 189–208.
• Cox, D. R. (1961). Tests of separate families of hypotheses. In Proc. 4th Berkeley Sympos. Math. Statist. and Prob., Vol. I 105–123. Univ. California Press, Berkeley, CA.
• Cox, D. R. and Hinkley, D. V. (1979). Theoretical Statistics. CRC Press, Boca Raton, FL.
• Cox, D. R. and Wermuth, N. (1990). An approximation to maximum likelihood estimates in reduced models. Biometrika 77 747–761.
• Creel, M. D. and Kristensen, D. (2013). Indirect likelihood inference. Technical report, Autonomous Univ. Barcelona.
• Davison, A. C. (2003). Statistical Models. Cambridge Series in Statistical and Probabilistic Mathematics 11. Cambridge Univ. Press, Cambridge.
• Denham, D. A., Ponnudurai, T., Nelson, G. S., Guy, F. and Rogers, R. (1972). Studies with Brugia pahangi. I. Parasitological observations on primary infections of cats (Felis catus). International Journal for Parasitology 2 239–247.
• Diggle, P. J. and Gratton, R. J. (1984). Monte Carlo methods of inference for implicit statistical models. J. R. Stat. Soc. Ser. B Stat. Methodol. 46 193–227. With discussion.
• Drovandi, C. C. (2012). Bayesian algorithms with applications. Ph.D. thesis, Queensland Univ. Technology, Brisbane.
• Drovandi, C. C. and Pettitt, A. N. (2011). Likelihood-free Bayesian estimation of multivariate quantile distributions. Comput. Statist. Data Anal. 55 2541–2556.
• Drovandi, C. C., Pettitt, A. N. and Faddy, M. J. (2011). Approximate Bayesian computation using indirect inference. J. R. Stat. Soc. Ser. C. Appl. Stat. 60 317–337.
• Drovandi, C. C., Pettitt, A. N. and Lee, A. (2015). Supplement to “Bayesian indirect inference using a parametric auxiliary model.” DOI:10.1214/14-STS498SUPP.
• Gallant, A. R. and McCulloch, R. E. (2009). On the determination of general scientific models with application to asset pricing. J. Amer. Statist. Assoc. 104 117–131.
• Gallant, A. R. and Tauchen, G. (1996). Which moments to match? Econometric Theory 12 657–681.
• Gillespie, D. T. (1977). Exact stochastic simulation of coupled chemical reactions. The Journal of Physical Chemistry 81 2340–2361.
• Gleim, A. and Pigorsch, C. (2013). Approximate Bayesian computation with indirect summary statistics. Technical report, Univ. Bonn.
• Gourieroux, C., Monfort, A. and Renault, E. (1993). Indirect inference. J. Appl. Econometrics 8 S85–S118.
• Heggland, K. and Frigessi, A. (2004). Estimating functions in indirect inference. J. R. Stat. Soc. Ser. B Stat. Methodol. 66 447–462.
• Jiang, W. and Turnbull, B. (2004). The indirect method: Inference based on intermediate statistics—A synthesis and examples. Statist. Sci. 19 239–263.
• Marjoram, P., Molitor, J., Plagnol, V. and Tavaré, S. (2003). Markov chain Monte Carlo without likelihoods. Proc. Natl. Acad. Sci. USA 100 15324–15328.
• Michael, E., Grenfell, B. T., Isham, V. S., Denham, D. A. and Bundy, D. A. P. (1998). Modelling variability in lymphatic filariasis: Macro filarial dynamics in the Brugia pahangi cat model. Proc. Roy. Soc. Lond. Ser. B 265 155–165.
• Møller, J., Pettitt, A. N., Reeves, R. and Berthelsen, K. K. (2006). An efficient Markov chain Monte Carlo method for distributions with intractable normalising constants. Biometrika 93 451–458.
• Murray, I., Ghahramani, A. and MacKay, D. (2006). MCMC for doubly-intractable distributions. In Proceedings of the 22nd Annual Conference on Uncertainty in Artificial Intelligence. AUAI Press, Arlington, VA.
• Rayner, G. D. and MacGillivray, H. L. (2002). Numerical maximum likelihood estimation for the $g$-and-$k$ and generalized $g$-and-$h$ distributions. Stat. Comput. 12 57–75.
• Reeves, R. W. and Pettitt, A. N. (2005). A theoretical framework for approximate Bayesian computation. In Proceedings of the 20th International Workshop on Statistical Modelling (A. R. Francis, K. M. Matawie, A. Oshlack and G. K. Smyth, eds.) 393–396. Univ. Western Sydney, Sydney, Australia.
• Riley, S., Donnelly, C. A. and Ferguson, N. M. (2003). Robust parameter estimation techniques for stochastic within-host macroparasite models. J. Theoret. Biol. 225 419–430.
• Scheffé, H. (1947). A useful convergence theorem for probability distributions. Ann. Math. Statistics 18 434–438.
• Smith, A. A. Jr. (1993). Estimating nonlinear time-series models using simulated vector autoregressions. J. Appl. Econometrics 8 S63–S84.
• Suswillo, R. R., Denham, D. A. and McGreevy, P. B. (1982). The number and distribution of Brugia pahangi in cats at different times after a primary infection. Acta Trop. 39 151–156.
• Wood, S. N. (2010). Statistical inference for noisy nonlinear ecological dynamic systems. Nature 466 1102–1104.

#### Supplemental materials

• Supplementary material: Supplement to “Bayesian Indirect Inference Using a Parametric Auxiliary Model”. This material contains a simple example to supplement Section 3.1 and additional information and results to supplement the examples in Sections 7.2 and 7.3.