The Annals of Applied Probability

Estimating functions evaluated by simulation: a Bayesian/analytic approach

J. R. Koehler, A. A. Puhalskii, and B. Simon

Full-text: Open access


Consider a function $f: B \to R_'$, where B is a compact subset of $R^m$ and consider a "simulation" used to estimate $f(x), x \epsilon B$ with the following properties: the simulation can switch from one $x \epsilon B$ to another in zero time, and a simulation at x lasting t units of time yields a random variable with mean $f(x)$ and variance $v(x)/t$ . With such a simulation we can divide T units of time into as many separate simulations as we like. Therefore, in principle we can design an "experiment" that spends $\tau(A)$ units of time simulating points in each $A \epsilon \mathscr{B}$, where $\mathscr{B}$ is the Borel $\sigma$-field on B and $\tau$ is an arbitrary finite measure on $(B, \mathscr{B})$. We call a design specified by a measure $\tau$ a "generalized design." We propose an approximation for f based on the data from a generalized design. When $\tau$ is discrete, the approximation, $\hat{f}$, reduces to a "Kriging"-like estimator. We study discrete designs in detail, including asymptotics (as the length of the simulation increases) and a numerical procedure for finding optimal n-point designs based on a Bayesian interpretation of $\hat{f}$ . Our main results, however, concern properties of generalized designs. In particular, we give conditions for integrals of $\hat{f}$ to be consistent estimates of the corresponding integrals of f. These conditions are satisfied for a large class of functions, f , even when $v(x)$ is not known exactly. If f is continuous and $\tau$ has a density, then consistent estimation of $f(x), x \epsilon B$ is also possible. Finally, we use the Bayesian interpretation of $\hat{f}$ to derive a variational problem satisfied by globally optimal designs. The variational problem always has a solution and we describe a sequence of n-point designs that approach (with respect to weak convergence) the set of globally optimal designs. Optimal designs are calculated for some generic examples. Our numerical studies strongly suggest that optimal designs have smooth densities.

Article information

Ann. Appl. Probab., Volume 8, Number 4 (1998), 1184-1215.

First available in Project Euclid: 9 August 2002

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62K05: Optimal designs 60G60: Random fields 60F25: $L^p$-limit theorems

Kriging experimental designs


Koehler, J. R.; Puhalskii, A. A.; Simon, B. Estimating functions evaluated by simulation: a Bayesian/analytic approach. Ann. Appl. Probab. 8 (1998), no. 4, 1184--1215. doi:10.1214/aoap/1028903377.

Export citation


  • Billingsley, P. (1968). Convergence of Probability Measures. Wiley, New York.
  • Currin, C., Mitchell, T., Morris, M. and Ylvisaker, D. (1991). Bayesian prediction of deterministic functions, with applications to the design and analysis of computer experiments. J. Amer. Statist. Assoc. 86 953-963.
  • Dudley, R. M. (1989). Real Analy sis and Probability. Wadsworth & Brooks/Cole, Pacific Grove, CA.
  • Hammersley, J. M. and Handscomb, D. C. (1964). Monte Carlo Methods. Methuen, London.
  • Hutson, V. and Py m, J. S. (1980). Applications of Functional Analy sis and Operator Theory. Academic Press, London.
  • Iglehart, D. L. (1978). The regenerative method for simulation analysis. In Current Trends in Programming Methodology-Software Modeling (K. M. Chandy and R. T. Yeh, eds.). Prentice-Hall, Englewood Cliffs, NJ.
  • Kiefer, J. and Wolfowitz, J. (1959). "Optimum designs in regression problems. Ann. Math. Statist. 30 289-312.
  • Koehler, J. R. and Owen, A. B. (1996). Computer experiments. In Handbook of Statistics (S. Ghosh and C. R. Rao, eds.) 13 261-308. North-Holland, Amsterdam.
  • Plaskota, L. (1992). Function approximation and integration on the Wiener space with noisy data. J. Complexity 8 301-323.
  • Pukelsheim, F. (1993). Optimal Design of Experiments. Wiley, New York.
  • Ripley, B. D. (1981). Spatial Statistics. Wiley, New York.
  • Ritter, K., Wasilkowski, G. W. and Wozniakowski, H. (1995). Multivariate integration and approximation for random fields satisfying Sacks-Ylvisaker conditions. Ann. Appl. Probab. 5 518-540.
  • Sacks, J., Schiller, S. B. and Welch, W. J. (1989). Designs for computer experiments. Technometrics 1 41-47.
  • Sacks, J., Welch, W. J., Mitchell, T. J. and Wy nn, H. P. (1989). Design and analysis of computer experiments. Statist. Sci. 4 409-423.
  • Searle, S. R. (1982). Matrix Algebra Useful for Statistics. Wiley, New York.
  • Shiry ayev, A. N. (1984). Probability. Springer, New York.
  • Simon, B. (1997). Convergence of `natural' rootfinding procedures. Unpublished manuscript.
  • Stein, M. L. (1988). Asy mptotically efficient prediction of a random field with a misspecified covariance function. Ann. Statist. 16 55-63.
  • Tricomi, F. G. (1985). Integral Equations. Dover, New York.