Bayesian Analysis

A dynamic modelling strategy for Bayesian computer model emulation

Fei Liu and Mike West

Full-text: Open access

Abstract

Computer model evaluation studies build statistical models of deterministic simulation-based predictions of field data to then assess and criticize the computer model and suggest refinements. Computer models are often expensive computationally: statistical models that adequately emulate their key features can be very much more efficient. Gaussian process models are often used as emulators, but the resulting computations lack the ability to scale to higher-dimensional, time-dependent or functional outputs. For some such problems, especially for contexts of time series outputs, building emulators using dynamic linear models provides a computationally attractive alternative as well as a flexible modelling approach capable of emulating a broad range of stochastic structures underlying the input-output simulations. We describe this here, combining Bayesian multivariate dynamic linear models with Gaussian process modelling in an effective manner, and illustrate the approach with data from a hydrological simulation model. The general strategy will be useful for other computer model evaluation studies with time series or functional outputs.

Article information

Source
Bayesian Anal., Volume 4, Number 2 (2009), 393-411.

Dates
First available in Project Euclid: 22 June 2012

Permanent link to this document
https://projecteuclid.org/euclid.ba/1340370283

Digital Object Identifier
doi:10.1214/09-BA415

Mathematical Reviews number (MathSciNet)
MR2507369

Zentralblatt MATH identifier
1330.65034

Keywords
Computer model emulation Dynamic linear model Forwarding filtering backward sampling Gaussian process Markov chain Monte Carlo Time-Varying Autoregression

Citation

Liu, Fei; West, Mike. A dynamic modelling strategy for Bayesian computer model emulation. Bayesian Anal. 4 (2009), no. 2, 393--411. doi:10.1214/09-BA415. https://projecteuclid.org/euclid.ba/1340370283


Export citation

References

  • Aguilar, O., Prado, R., Huerta, G., and West, M. (1999). "Bayesian inference on latent structure in time series (with discussion)." In Bernardo, J., Berger, J., Dawid, A., and Smith, A. (eds.), Bayesian Statistics 6, 3–26. Oxford University Press.
  • Bayarri, M., Berger, J., Garcia-Donato, G., Liu, F., Palomo, J., Paulo, R., Sacks, J., Walsh, D., Cafeo, J., and Parthasarathy, R. (2007). "Computer Model Validation with Functional Outputs." Annals of Statistics, 35: 1874–1906.
  • Bayarri, M. J., Berger, J. O., Kennedy, M., Kottas, A., Paulo, R., Sacks, J., Cafeo, J. A., Lin, C. H., and Tu, J. (2005). "Bayesian Validation of a Computer Model for Vehicle Crashworthiness." Technical report, National Institute of Statistical Sciences, RTP, NC, USA. http://www.niss.org/technicalreports.html.
  • Bayarri, M. J., Berger, J. O., Paulo, R., Sacks, J., Cafeo, J. A., Cavendish, J., Lin, C.-H., and Tu, J. (2007). "A Framework for Validation of Computer Models." Technometrics, 49(2): 138–154.
  • Carter, C. K. and Kohn, R. (1994). "On Gibbs sampling for state-space models." Biometrika, 81: 541–553.
  • Craig, P. S., Goldstein, M., Rougier, J., and Seheult, A. (2001). "Bayesian forecasting for complex systems using computer simulators." Journal of the American Statistical Association, 96(454): 717–729.
  • Currin, C., Mitchell, T., Morris, M., and Ylvisaker, D. (1991). "Bayesian prediction of deterministic functions, with applications to the design and analysis of computer experiments." Journal of the American Statistical Association, 86: 953–963.
  • Frühwirth-Schnatter, S. (1994). "Data augmentation and dynamic linear models." Journal of Time Series Analysis, 15: 183–202.
  • Fuentes, M., Guttorp, P., and Challenor, P. (2003). "Statistical assessment of numerical models." International Statistical Review, 201–221.
  • Gelfand, A. E. and Smith, A. F. M. (1990). "Sampling based approaches to calculating marginal densities." Journal of the American Statistics Association, 85: 398–409.
  • Goldstein, M. and Rougier, J. C. (2003). "Calibrated Bayesian forecasting using large computer simulators." Technical report, University of Durham, UK, http://www.maths.dur.ac.uk/stats/physpred/papers/CalibratedBayesian.ps.
  • –- (2005). "Probabilistic formulations for transferring inferences from mathematical models to physical systems." SIAM Journal on Scientific Computing, 26(2): 467–487.
  • Higdon, D., Gattiker, J., Williams, B., and Rightley, M. (2008). "Computer Model Calibration Using High-Dimensional Output." Journal of the American Statistical Association, 103(482): 570–583.
  • Higdon, D., Kennedy, M., Cavendish, J., Cafeo, J., and Ryne, R. D. (2004). "Combining field observations and simulations for calibration and prediction." SIAM Journal of Scientific Computing, 26: 448–466.
  • Higdon, D., Nakhleh, C., Gattiker, J., and Williams, B. (2008). "A Bayesian calibration approach to the thermal Problem." Computer Methods in Applied Mechanics and Engineering (CMAME), 197: 2431–2441.
  • Higdon, D., Williams, B., Moore, L., McKay, M., and Keller-McNulty, S. (2004). "Uncertainty Quantification for Combining Experimental Data and Computer Simulations." Technical Report LA-UR 04-6562, Los Alamos National Laboratories, USA.
  • Kennedy, M. C. and O'Hagan, A. (2001). "Bayesian calibration of computer models (with discussion)." Journal of the Royal Statistical Society B, 63: 425–464.
  • Kennedy, M. C., O'Hagan, A., and Higgins, N. (2002). "Bayesian analysis of computer code outputs." In Quantitative Methods for Current Environmental Issues. C. W. Anderson, V. Barnett, P. C. Chatwin, and A. H. El-Shaarawi (eds.), 227–243. Springer-Verlag: London.
  • Kuczera, G., Kavetski, D., Franks, S., and Thyer, M. (2006). "Towards a Bayesian total error analysis of conceptual rainfall-runoff models: Characterising model error using storm-dependent parameters." Journal of Hydrology, 331: 161–177.
  • Morris, M. D., Mitchell, T. J., and Ylvisaker, D. (1993). "Bayesian design and analysis of computer experiments: Use of derivatives in surface prediction." Technometrics, 35: 243–255.
  • Paulo, R., Lin, J., Rouphail, N., and Sacks, J. (2005). "Calibrating and Validating Deterministic Traffic Models:Application to the HCM" Control Delay at Signalized Intersections. Transportation Research Record: Journal of the Transportation Research Board, 1920: 95–105.
  • Prado, R., Huerta, G., and West, M. (2001). "Bayesian time-varying autoregressions: Theory, methods and applications." Resenhas, 4: 405–422.
  • Prado, R. and West, M. (1997). "Exploratory modelling of multiple non-stationary time series: Latent process structure and decompositions." In Gregoire, T., Brillinger, D., Diggle, P., Russek-Cohen, E., Warren, W., and Wolfinger, R. (eds.), Modelling Longitudinal and Spatially Correlated Data, 349–362. New York: Springer-Verlag.
  • Reichert, P., White, G., Bayarri, M., Pitman, E., and Santer, T. (2008). "Mechanism-based Emulation of Dynamic Simulators: Concept and Application in Hydrology." http://www.eawag.ch/kuerze/personen/homepages/reichert/index_EN.
  • Rougier, J. (2008). "Efficient Emulators for Multivariate Deterministic Functions." Journal of Computational and Graphical Statistics (In press).
  • –- (2008). "Formal Bayes Methods for Model Calibration with Uncertainty." In Beven, K. and Hall, J. (eds.), Applied Uncertainty Analysis for Flood Risk Management (In press). Imperial College Press/World Scientific. Draft version available at http://www.maths.bris.ac.uk/~mazjcr/FRMbox2-4.pdf.
  • Rougier, J., Sexton, D., Murphy, J., and Stainforth, D. (2008). "Analysing the climate sensitivity of the HadSM3 climate model using ensembles from different but related experiments." Technical report, University of Bristol, UK. Available at http://www.maths.bris.ac.uk/~mazjcr/qump1.pdf.
  • Sacks, J., Welch, W. J., Mitchell, T. J., and Wynn, H. P. (1989). "Design and analysis of computer experiments (C/R: p423-435)." Statistical Science, 4: 409–423.
  • Sanso, B., Forest, C., and Zantedeschi, D. (2007). "Statistical Calibration of Climate System Properties." Technical Report asm2007-06, Dept. of Applied Math & Statistics, University of California, Santa Cruz, http://www.soe.ucsc.edu/research/report?ID=476.
  • –- (2008). "Inferring Climate System Properties Using a Computer Model." Bayesian Analysis, 3(1): 1–38.
  • Santner, T., Williams, B., and Notz, W. (2003). The Design and Analysis of Computer Experiments. New York: Springer-Verlag.
  • Tebaldi, C. and Sanso, B. (2008). "Joint Projections of Temperature and Precipitation Change from Multiple Climate Models: A Hierarchical Bayes Approach." Journal of the Royal Statistical Society: Series A (In press).
  • Welch, W. J., Buck, R. J., Sacks, J., Wynn, H. P., Mitchell, T. J., and Morris, M. D. (1992). "Screening, predicting, and computer experiments." Technometrics, 34: 15–25.
  • West, M. and Harrison, P. (1997). Bayesian Forecasting and Dynamic Models. New York: Springer-Verlag, 2 edition.