The Annals of Applied Statistics

Parameter estimation for computationally intensive nonlinear regression with an application to climate modeling

Dorin Drignei, Chris E. Forest, and Doug Nychka

Full-text: Open access


Nonlinear regression is a useful statistical tool, relating observed data and a nonlinear function of unknown parameters. When the parameter-dependent nonlinear function is computationally intensive, a straightforward regression analysis by maximum likelihood is not feasible. The method presented in this paper proposes to construct a faster running surrogate for such a computationally intensive nonlinear function, and to use it in a related nonlinear statistical model that accounts for the uncertainty associated with this surrogate. A pivotal quantity in the Earth’s climate system is the climate sensitivity: the change in global temperature due to doubling of atmospheric CO2 concentrations. This, along with other climate parameters, are estimated by applying the statistical method developed in this paper, where the computationally intensive nonlinear function is the MIT 2D climate model.

Article information

Ann. Appl. Stat., Volume 2, Number 4 (2008), 1217-1230.

First available in Project Euclid: 8 January 2009

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Equilibrium climate sensitivity observed and modeled climate space–time modeling statistical surrogate temperature data


Drignei, Dorin; Forest, Chris E.; Nychka, Doug. Parameter estimation for computationally intensive nonlinear regression with an application to climate modeling. Ann. Appl. Stat. 2 (2008), no. 4, 1217--1230. doi:10.1214/08-AOAS210.

Export citation


  • Andronova, N. G. and Schlesinger, M. E. (2001). Objective estimation of the probability density function for climate sensitivity. J. Geophys. Res. 106 22,605–22,612.
  • Annan, J. D. and Hargreaves, J. C. (2006). Using multiple observationally-based constraints to estimate climate sensitivity. Geophys. Res. Let. 33 L06704. DOI: 10.1029/2005GL025259.
  • Bayarri, M. J., Walsh, D., Berger, J. O., Cafeo, J., Garcia-Donato, G., Liu, F., Palomo, J., Parthasarathy, R. J., Paulo, R. and Sacks, J. (2007). Computer model validation with functional output. Ann. Statist. 35 1874–1906.
  • Craig, P. S., Goldstein, M., Rougier, J. C. and Seheult, A. H. (2001). Bayesian forecasting for complex systems using computer simulators. J. Amer. Statist. Assoc. 96 717–729.
  • Currin, C., Mitchell, T., Morris, M. and Ylvisaker, D. (1991). Bayesian prediction of deterministic functions, with applications to the design and analysis of computer experiments. J. Amer. Statist. Assoc. 86 953–963.
  • Drignei, D. (2006). Empirical Bayesian analysis for high-dimensional computer output. Technometrics 48 230–240.
  • Efron, B. and Tibshirani, R. J. (1993). An Introduction to the Bootstrap. Chapman and Hall, New York.
  • Forest, C. E., Stone, P. H., Sokolov, A. P., Allen, M. R. and Webster, M. D. (2002). Quantifying uncertainties in climate system properties with the use of recent climate observations. Science 295 113–117.
  • Forest, C. E., Nychka, D., Sanso, B. and Tebaldi, C. (2003). Towards a rigorous MCMC estimation of PDFs of Climate System Properties. Eos Trans. AGU Fall Meeting Suppl. 84 Abstract GC31B-0196.
  • Forest, C. E., Stone, P. H. and Sokolov, A. P. (2006). Estimated PDFs of climate system properties including natural and anthropogenic forcings. Geophys. Res. Let. 33 L01705. DOI: 10.1029/2005GL023977.
  • Gregory, J. M., Stouffer, R. J., Raper, S. C. B., Stott, P. A. and Rayner, N. A. (2002). An observationally based estimate of the climate sensitivity. J. Climate 15 3117–3121.
  • Hansen, J., Russell, G., Rind, D., Stone, P., Lacis, A., Lebedeff, S., Ruedy, R. and Travis, L. (1983). Efficient three-dimensional global models for climate studies: Models I and II. Mon. Weath. Rev. 111 609–662.
  • Hansen, J., Lacis, A., Rind, D., Russell, G., Stone, P., Fung, I., Ruedy, R. and Lerner, J. (1984). Climate sensitivity: Analysis of feedback mechanisms. Climate processes and climate sensitivity. In Geophysical Monograph (J. E. Hansen and T. Takahashi, eds.) 29 130–163. American Geophysical Union, Washington, DC.
  • Hegerl, G. C., Crowley, T. J., Hyde, W. T. and Frame, D. J. (2006). Climate sensitivity constrained by temperature reconstructions over the past seven centuries. Nature 440. DOI: 10.1038/nature04679.
  • Henderson-Sellers, A. and McGuffie, K. (1987). A Climate Modelling Primer, 1st ed. Wiley, Chichester.
  • Higdon, D., Gattiker, J., Williams, B. and Rightley, M. (2008). Computer model calibration using high dimensional outputs. J. Amer. Statist. Assoc. 103 570–583.
  • Jones, P. D. (1994). Hemispheric surface air temperature variations: A reanalysis and an update to 1993. J. Climate 7 1794–1802.
  • Kennedy, M. C. and O’Hagan, A. (2001). Bayesian calibration of computer models. J. Roy. Statist. Soc. Ser. B 63 425–450.
  • Knutti, R., Stocker, T. F., Joos, F. and Plattner, G.-K. (2002). Constraints on radiative forcing and future climate change from observations and climate model ensembles. Nature 416 719–723.
  • Knutti, R., Stocker, T. F., Joos, F. and Plattner, G.-K. (2003). Probabilistic climate change projections using neural networks. Clim. Dyn. 21 257–272.
  • Levitus, S., Antonov, J., Boyer, T. P. and Stephens, C. (2000). Warming of the World Ocean. Science 287 2225–2229.
  • National Research Council (NRC) (1979). Carbon Dioxide and Climate: A Scientific Assessment. National Academy Press, Washington, DC.
  • Nychka, D., Wikle, C. and Royle, J. A. (2002). Multiresolution models for nonstationary spatial covariance functions. Statist. Modelling 2 315–332.
  • Parker, D. E., Gordon, M., Cullum, D. P. N., Sexton, D. M. H., Folland, C. K. and Rayner, N. (1997). A new global gridded radiosonde temperature data base and recent temperature trends. Geophys. Res. Lett. 24 1499–1502.
  • Sacks, J., Welch, W. J., Mitchell, T. J. and Wynn, H. P. (1989). Design and analysis of computer experiments. Statist. Sci. 4 409–423.
  • Santner, T. J., Williams, B. J. and Notz, W. I. (2003). The Design and Analysis of Computer Experiments. Springer, New York.
  • Sanso, B., Forest, C. E. and Zantedeschi, D. (2008). Inferring climate system properties using a computer model. Bayesian Anal. 3 1–38.
  • Schlesinger, M. E. and Mitchell, J. F. B. (1987). Climate model simulations of the equilibrium climatic response to increased carbon dioxide. Rev. Geophys. 25 760–798.
  • Seber, G. A. F. and Wild, C. J. (1989). Nonlinear Regression. Wiley, New York.
  • Sokolov, A. P. and Stone, P. H. (1998). A flexible climate model for use in integrated assessments. Clim. Dyn. 14 291–303.
  • Stein, M. L. (1999). Interpolation of Spatial Data: Some Theory for Kriging. Springer, New York.
  • Stone, P. H. and Yao, M. S. (1987). Development of a two-dimensional zonally averaged statistical–dynamical model. II: The role of eddy momentum fluxes in the general circulation and their parametrization. J. Atmos. Sci. 44 3769–3786.
  • Stone, P. H. and Yao, M. S. (1990). Development of a two-dimensional zonally averaged statistical–dynamical model. III: Parametrization of the eddy fluxes of heat and moisture. J. Climate 3 726–740.