The Annals of Statistics

Construction of nested space-filling designs

Peter Z. G. Qian, Mingyao Ai, and C. F. Jeff Wu

Full-text: Open access


New types of designs called nested space-filling designs have been proposed for conducting multiple computer experiments with different levels of accuracy. In this article, we develop several approaches to constructing such designs. The development of these methods also leads to the introduction of several new discrete mathematics concepts, including nested orthogonal arrays and nested difference matrices.

Article information

Ann. Statist., Volume 37, Number 6A (2009), 3616-3643.

First available in Project Euclid: 17 August 2009

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62K15: Factorial designs
Secondary: 62K20: Response surface designs

Computer experiments design of experiments difference matrices orthogonal arrays OA-based Latin hypercubes randomized orthogonal arrays Wang–Wu method


Qian, Peter Z. G.; Ai, Mingyao; Wu, C. F. Jeff. Construction of nested space-filling designs. Ann. Statist. 37 (2009), no. 6A, 3616--3643. doi:10.1214/09-AOS690.

Export citation


  • Addelman, S. and Kempthorne, O. (1961). Some main-effect plans and orthogonal arrays of strength two. Ann. Math. Statist. 32 1167–1176.
  • Bayarri, M. J., Berger, J. O., Cafeo, J., Garcia-Donato, G., Liu, F., Palomo, J., Parthasarathy, R. J., Paulo, R., Sacks, J. and Walsh, D. (2007). Computer model validation with functional output. Ann. Statist. 35 1874–1906.
  • Bose, R. C. and Bush, K. A. (1952). Orthogonal arrays of strength two and three. Ann. Math. Statist. 23 508–524.
  • Butler, N. A. (2001). Optimal and orthogonal Latin hypercube designs for computer experiments. Biometrika 88 847–857.
  • Dalal, S. R. and Mallows, C. L. (1998). Factor-covering designs for testing software. Technometrics 40 234–243.
  • Fang, K. T., Li, R. Z. and Sudjianto, A. (2006). Design and Modeling for Computer Experiments. Chapman & Hall, New York.
  • Fang, K. T., Lin, D. K. J., Winker, P. and Zhang, Y. (2000). Uniform design: Theory and application. Technometrics 42 237–248.
  • Goldstein, M. and Rougier, J. (2004). Probabilistic formulations for transferring inferences from mathematical models to physical systems. SIAM J. Sci. Comput. 26 467–487.
  • Hedayat, A. S., Sloane, N. J. A. and Stufken, J. (1999). Orthogonal Arrays: Theory and Applications. Springer, New York.
  • Hedayat, A. S., Stufken, J. and Su, G. (1996). On difference schemes and orthogonal arrays of strength t. J. Statist. Plann. Inference 56 307–324.
  • Higdon, D., Kennedy, M. C., Cavendish, J. C., Cafeo, J. A. and Ryne, R. D. (2004). Combining field data and computer simulations for calibration and prediction. SIAM J. Sci. Comput. 26 448–466.
  • Johnson, D. M., Dulmage, A. L. and Mendelsohn, N. S. (1961). Orthomorphisms of groups and orthogonal Latin squares. I. Canad. J. Math. 13 356–372.
  • Kennedy, M. C. and O’Hagan, A. (2000). Predicting the output from a complex computer code when fast approximations are available. Biometrika 87 1–13.
  • Kennedy, M. C. and O’Hagan, A. (2001). Bayesian calibration of computer models. J. R. Stat. Soc. Ser. B Stat. Methodol. 63 425–464.
  • Lam, R. L. H., Welch, W. J. and Young, S. S. (2002). Uniform coverage designs for molecule selection. Technometrics 44 99–109.
  • Leary, S., Bhaskar, A. and Keane, A. (2003). Optimal orthogonal-array-based Latin hypercubes. J. Appl. Statist. 30 585–598.
  • Loh, W. L. (1996a). A combinatorial central limit theorem for randomized orthogonal array sampling designs. Ann. Statist. 24 1209–1224.
  • Loh, W. L. (1996b). On Latin hypercube sampling. Ann. Statist. 24 2058–2080.
  • Loh, W. L. (2008). A multivariate central limit theorem for randomized orthogonal array sampling designs in computer experiments. Ann. Statist. 36 1983–2023.
  • McKay, M. D., Beckman, R. J. and Conover, W. J. (1979). A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics 21 239–245.
  • Niederreiter, H. (1992). Random Number Generation and Quasi-Monte Carlo Methods. SIAM, Philadelphia, PA.
  • Oberkampf, W.L. and Trucano, T. (2007). Verification and Validation Benchmarks. Sandia National Laboratories (SAND 2007-0853), Albuquerque, New Mexico.
  • Owen, A. B. (1992). Orthogonal arrays for computer experiments, integration and visualization. Statist. Sinica 2 439–452.
  • Owen, A. B. (1994a). Lattice sampling revisited: Monte Carlo variance of means over randomized orthogonal arrays. Ann. Statist. 22 930–945.
  • Owen, A. B. (1994b). Controlling correlations in Latin hypercube samples. J. Amer. Statist. Assoc. 88 1517–1522.
  • Qian, Z., Seepersad, C. C., Roshan, V. R., Allen, J. K. and Wu, C. F. J. (2006). Building surrogate models based on detailed and approximate simulations. ASME Transactions: J. Mechanical Design 128 668–677.
  • Qian, P. Z. G., Tang, B. and Wu, C. F. J. (2009). Nested space-filling designs for computer experiments with two levels of accuracy. Statist. Sinica 19 287–300.
  • Qian, P. Z. G. and Wu, C. F. J. (2008). Bayesian hierarchical modeling for integrating low-accuracy and high-accuracy experiments. Technometrics 50 192–204.
  • Reese, C. S., Wilson, A. G., Hamada, M., Martz, H. F. and Ryan, K. J. (2004). Integrated analysis of computer and physical experiments. Technometrics 46 153–164.
  • Sacks, J., Welch, W. J., Mitchell, T. J. and Wynn, H. P. (1989). Design and analysis of computer experiments. Statist. Sci. 4 409–435.
  • Santner, T. J., Williams, B. J. and Notz, W. I. (2003). The Design and Analysis of Computer Experiments. Springer, New York.
  • Seberry, J. (1979). Some remarks on generalised Hadamard matrices and theorems of Rajkundlia on SBIBD’s. In Combinatorial Mathematics. VI (A. F. Horadam and W. D. Wallis, eds.) 154–164. Springer, Berlin.
  • Shrikhande, S. S. (1964). Generalized Hadamard matrices and orthogonal arrays of strength two. Canad. J. Math. 16 736–740.
  • Stein, M. (1987). Large sample properties of simulations using Latin hypercube sampling. Technometrics 29 143–151.
  • Steinberg, D. M. and Lin, D. K. J. (2006). A construction method for orthogonal Latin hypercube designs. Biometrika 93 279–288.
  • Tang, B. (1993). Orthogonal array-based Latin hypercubes. J. Amer. Statist. Assoc. 88 1392–1397.
  • Tang, B. (1994). A theorem for selecting OA-based Latin hypercubes using a distance criterion. Comm. Statist. Theory Methods 23 2047–2058.
  • Tang, B. (1998). Selecting Latin hypercubes using correlation criteria. Statist. Sinica 8 409–435.
  • Wang, J. C. (1996). Mixed difference matrices and the construction of orthogonal arrays. Statist. Probab. Lett. 28 121–126.
  • Wang, J. C. and Wu, C. F. J. (1991). An approach to the construction of asymmetrical orthogonal arrays. J. Amer. Statist. Assoc. 86 450–455.
  • Welch, W. J., Buck, R. J., Sacks, J., Wynn, H. P., Mitchell, T. J. and Morris, M. D. (1992). Screening, predicting and computer experiments. Technometrics 34 15–25.
  • Wu, C. F. J. and Hamada, M. (2000). Experiments: Planning, Analysis, and Parameter Design Optimization. Wiley, New York.
  • Ye, K. Q. (1998). Orthogonal column Latin hypercubes and their application in computer experiments. J. Amer. Statist. Assoc. 88 1392–1397.