Bayesian Analysis

Efficient MCMC for Climate Model Parameter Estimation: Parallel Adaptive Chains and Early Rejection

Antti Solonen, Pirkka Ollinaho, Marko Laine, Heikki Haario, Johanna Tamminen, and Heikki Järvinen

Full-text: Open access

Abstract

The emergence of Markov chain Monte Carlo (MCMC) methods has opened a way for Bayesian analysis of complex models. Running MCMC samplers typically requires thousands of model evaluations, which can exceed available computer resources when this evaluation is computationally intensive. We will discuss two generally applicable techniques to improve the efficiency of MCMC. First, we consider a parallel version of the adaptive MCMC algorithm of Haario et al. (2001), implementing the idea of inter-chain adaptation introduced by Craiu et al. (2009). Second, we present an early rejection (ER) approach, where model simulation is stopped as soon as one can conclude that the proposed parameter value will be rejected by the MCMC algorithm.

This work is motivated by practical needs in estimating parameters of climate and Earth system models. These computationally intensive models involve non-linear expressions of the geophysical and biogeochemical processes of the Earth system. Modeling of these processes, especially those operating in scales smaller than the model grid, involves a number of specified parameters, or ‘tunables’. MCMC methods are applicable for estimation of these parameters, but they are computationally very demanding. Efficient MCMC variants are thus needed to obtain reliable results in reasonable time. Here we evaluate the computational gains attainable through parallel adaptive MCMC and Early Rejection using both simple examples and a realistic climate model.

Article information

Source
Bayesian Anal. Volume 7, Number 3 (2012), 715-736.

Dates
First available in Project Euclid: 28 August 2012

Permanent link to this document
https://projecteuclid.org/euclid.ba/1346158781

Digital Object Identifier
doi:10.1214/12-BA724

Mathematical Reviews number (MathSciNet)
MR2981633

Zentralblatt MATH identifier
1330.60091

Keywords
Adaptive MCMC Climate Models Parallel MCMC Early Rejection

Citation

Solonen, Antti; Ollinaho, Pirkka; Laine, Marko; Haario, Heikki; Tamminen, Johanna; Järvinen, Heikki. Efficient MCMC for Climate Model Parameter Estimation: Parallel Adaptive Chains and Early Rejection. Bayesian Anal. 7 (2012), no. 3, 715--736. doi:10.1214/12-BA724. https://projecteuclid.org/euclid.ba/1346158781


Export citation

References

  • Andrieu, C. and Moulines, E. (2006). “On the ergodicity properties of some adaptive MCMC algorithms.” Annals of Applied Probability, 16(3): 1462–1505.
  • Annan, J. D. and Hargreaves, J. C. (2007). “Efficient estimation and ensemble generation in climate modeling.” Philosophical Transactions of the Royal Society A, 365: 2077–2088.
  • Beskos, A., Papaspiliopoulos, O., and Roberts, G. (2006a). “Retrospective exact simulation of diffusion sample paths with applications.” Bernoulli, 12(6): 1077.
  • Beskos, A., Papaspiliopoulos, O., Roberts, G., and Fearnhead, P. (2006b). “Exact and computationally efficient likelihood-based estimation for discretely observed diffusion processes (with discussion).” Journal of the Royal Statistical Society: Series B (Statistical Methodology), 68(3): 333–382.
  • Brockwell, A. (2006). “Parallel Markov Chain Monte Carlo Simulation by Pre-fetching.” Journal of Computational and Graphical Statistics, 15(1): 246–26.
  • Brockwell, A. and Kadane, J. (2005). “Identification of regeneration times in MCMC simulation, with application to adaptive schemes.” Journal of Computational and Graphical Statistics, 14(2): 436–458.
  • Cappe, O., Guillin, A., Marin, J., and Robert, C. P. (2004). “Population Monte Carlo.” Journal of Computational and Graphical Statistics, 13: 907–929.
  • Christen, J. A. and Fox, C. (2005). “MCMC using an Approximation.” Journal of Computational and Graphical Statistics, 14(4): 795–810.
  • Cornuet, J., Marin, J., Mira, A., and Robert, C. (2012). “Adaptive Multiple Importance Sampling.” Scandinavian Journal of Statistics, available on-line in Early View. URL http://arxiv.org/abs/0907.1254
  • Craiu, R. V., Rosenthal, J., and Yang, C. (2009). “Learn From Thy Neighbor: Parallel-Chain and Regional Adaptive MCMC.” Journal of the American Statistical Association, 104(488): 1454–146.
  • Drignei, D., Forest, C. E., and Nychka, D. (2008). “Parameter estimation for computationally intensive nonlinear regression with an application to climate modeling.” The Annals of Applied Statistics, 2(4): 1217–1230.
  • Dunson, D. and Park, J. (2008). “Kernel stick-breaking processes.” Biometrika, 95(2): 307–323.
  • Gelfand, A. and Smith, A. (1990). “Sampling-Based Approaches to Calculating Marginal Densities.” Journal of the American Statistical Association, 85: 398–409.
  • Gelman, A. and Rubin, D. B. (1992). “Inference from Iterative Simulation Using Multiple Sequences.” Statistical Science, 7(4): 457–472.
  • Geyer, C. J. (1992). “Practical Markov Chain Monte Carlo.” Statistical Science, 7(4): 473–483.
  • Gilks, W., Roberts, G., and Sahu, S. (1998). “Adaptive Markov chain Monte Carlo through regeneration.” Journal of the American Statistical Association, 93(443): 1045–1054.
  • Haario, H., Laine, M., Mira, A., and Saksman, E. (2006). “DRAM: Efficient adaptive MCMC.” Statistics and Computing, 16(3): 339–354.
  • Haario, H., Saksman, E., and Tamminen, J. (1999). “Adaptive proposal distribution for random walk Metropolis algorithm.” Computational Statistics, 14: 375–395.
  • — (2001). “An adaptive Metropolis algorithm.” Bernoulli, 7(2): 223–242.
  • — (2005). “Componentwise adaptation for high dimensional MCMC.” Computational Statistics, 20(2): 265–273.
  • Hastings, W. K. (1970). “Monte Carlo sampling using Markov chains and their applications.” Biometrika, 57(1): 97–109.
  • Jackson, C. S. (2009). “Use of Bayesian inference and data to improve simulations of multi-physics climate phenomena.” Journal of Physics: Conference Series, 180. SciDAC 2009 14–18 June 2009, San Diego, California, USA.
  • Järvinen, H., Räisänen, P., Laine, M., Tamminen, J., Ilin, A., Oja, E., Solonen, A., and Haario, H. (2010). “Estimation of ECHAM5 climate model closure parameters with adaptive MCMC.” Atmospheric Chemistry and Physics, 10(20): 9993–10002. URL http://www.atmos-chem-phys.net/10/9993/2010/
  • Laine, M. (2008). Adaptive MCMC Methods with Applications in Environmental and Geophysical Models. Finnish Meteorological Institute Contributions, No. 6. URL http://www.helsinki.fi/mjlaine/mcmc/
  • Lorenz, E. N. (1963). “Deterministic nonperiodic flow.” Journal of the Atmospheric Sciences, 20: 130–141.
  • Malve, O., Laine, M., and Haario, H. (2005). “Estimation of winter respiration rates and prediction of oxygen regime in a lake using Bayesian inference.” Ecological Modelling, (182): 183–197.
  • Metropolis, N., Rosenbluth, A. W., Rosenbluth, M. N., Teller, A. H., and Teller, E. (1953). “Equation of State Calculations by Fast Computing Machines.” The Journal of Chemical Physics, 21(6): 1087–1092.
  • Mira, A. (2001). “On Metropolis-Hastings algorithms with delayed rejection.” Metron, LIX(3–4): 231–241.
  • Mykland, P., Tierney, L., and Yu, B. (1995). “Regeneration in Markov Chain Samplers.” Journal of the American Statistical Association, 90(429): 233–241.
  • Papaspiliopoulos, O. and Roberts, G. (2008). “Retrospective Markov chain Monte Carlo methods for Dirichlet process hierarchical models.” Biometrika, 95(1): 169–186.
  • Rantala, J. and Piche, R. (2009). “Software Systems for Distributed Scientific Computing.” Research Report 95, Tampere University of Technology. Department of Mathematics. URL http://math.tut.fi/en/wp-content/uploads/2009/10/report 95.pdf
  • Robert, C. P. and Casella, G. (2005). Monte Carlo Statistical Methods. New York: Springer, second edition.
  • Roberts, G. and Rosenthal, J. (2001). “Optimal scaling for various Metropolis-Hastings algorithms.” Statistical Science, 16(4): 351–367.
  • — (2007). “Coupling and Ergodicity of Adaptive MCMC.” Journal of Applied Probability, 44(2): 458–475.
  • Roeckner, E., Bäuml, G., Bonaventura, L., Brokopf, R., Esch, M., Giorgetta, M., Hagemann, S., Kirchner, I., Kornblueh, L., Manzini, E., Rhodin, A., Schlese, U., Schulzweida, U., and Tompkins, A. (2003). “The atmospheric general circulation model ECHAM5, Part I Model Description.” Technical Report No. 349, Max-Planck-Institut für Meteorologie.
  • Rosenthal, J. S. (2000). “Parallel Computing and Monte Carlo algorithms.” Far Eastern Journal of Theoretical Statistics, 4: 207–236.
  • Smith, T. J. and Marshall, L. A. (2008). “Bayesian methods in hydrologic modeling: A study of recent advancements in Markov chain Monte Carlo techniques.” Water Resources Research, 44(W00B05): 1–9.
  • Strid, I. (2009). “Efficient parallelisation of Metropolis-Hastings algorithms using a prefetching approach.” Computational Statistics and Data Analysis, 54(11): 2814–2835.
  • Vahteristo, K., Maury, S., Laari, A., Solonen, A., Haario, H., and Koskimies, S. (2009). “Kinetics of neopentyl glycol esterification with different carboxylic acids.” Industial and Engineering Chemistry Research, 48(13): 6237–6247.
  • Vihola, M. (2011). “Can the Adaptive Metropolis Algorithm Collapse Without the Covariance Lower Bound?” Electronic Journal of Probability, 16: 45–75.
  • Villagran, A., Huerta, G., Jackson, C. S., and Sen, M. K. (2008). “Computational Methods for Parameter Estimation in Climate Models.” Bayesian Analysis, 3(3): 1–27.