The Annals of Statistics

Optimal design for linear models with correlated observations

Holger Dette, Andrey Pepelyshev, and Anatoly Zhigljavsky

Full-text: Open access


In the common linear regression model the problem of determining optimal designs for least squares estimation is considered in the case where the observations are correlated. A necessary condition for the optimality of a given design is provided, which extends the classical equivalence theory for optimal designs in models with uncorrelated errors to the case of dependent data. If the regression functions are eigenfunctions of an integral operator defined by the covariance kernel, it is shown that the corresponding measure defines a universally optimal design. For several models universally optimal designs can be identified explicitly. In particular, it is proved that the uniform distribution is universally optimal for a class of trigonometric regression models with a broad class of covariance kernels and that the arcsine distribution is universally optimal for the polynomial regression model with correlation structure defined by the logarithmic potential. To the best knowledge of the authors these findings provide the first explicit results on optimal designs for regression models with correlated observations, which are not restricted to the location scale model.

Article information

Ann. Statist., Volume 41, Number 1 (2013), 143-176.

First available in Project Euclid: 5 March 2013

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62K05: Optimal designs 60G10: Stationary processes
Secondary: 31A10: Integral representations, integral operators, integral equations methods 45C05: Eigenvalue problems [See also 34Lxx, 35Pxx, 45P05, 47A75]

Optimal design correlated observations integral operator eigenfunctions arcsine distribution logarithmic potential


Dette, Holger; Pepelyshev, Andrey; Zhigljavsky, Anatoly. Optimal design for linear models with correlated observations. Ann. Statist. 41 (2013), no. 1, 143--176. doi:10.1214/12-AOS1079.

Export citation


  • Bickel, P. J. and Herzberg, A. M. (1979). Robustness of design against autocorrelation in time. I. Asymptotic theory, optimality for location and linear regression. Ann. Statist. 7 77–95.
  • Bickel, P. J., Herzberg, A. M. and Schilling, M. F. (1981). Robustness of design against autocorrelation in time. II. Optimality, theoretical and numerical results for the first-order autoregressive process. J. Amer. Statist. Assoc. 76 870–877.
  • Boltze, L. and Näther, W. (1982). On effective observation methods in regression models with correlated errors. Math. Operationsforsch. Statist. Ser. Statist. 13 507–519.
  • Dette, H., Kunert, J. and Pepelyshev, A. (2008). Exact optimal designs for weighted least squares analysis with correlated errors. Statist. Sinica 18 135–154.
  • Dette, H., Pepelyshev, A. and Zhigljavsky, A. (2008). Improving updating rules in multiplicative algorithms for computing $D$-optimal designs. Comput. Statist. Data Anal. 53 312–320.
  • Dette, H., Pepelyshev, A. and Holland-Letz, T. (2010). Optimal designs for random effect models with correlated errors with applications in population pharmacokinetics. Ann. Appl. Stat. 4 1430–1450.
  • Efromovich, S. (1999). Nonparametric Curve Estimation: Methods, Theory, and Applications. Springer, New York.
  • Efromovich, S. (2008). Optimal sequential design in a controlled non-parametric regression. Scand. J. Stat. 35 266–285.
  • Fahmy, M. H., Abdou, M. A. and Darwish, M. A. (1999). Integral equations and potential-theoretic type integrals of orthogonal polynomials. J. Comput. Appl. Math. 106 245–254.
  • Gradshteyn, I. S. and Ryzhik, I. M. (1965). Table of Integrals, Series, and Products. Academic Press, New York.
  • Grenander, U. (1950). Stochastic processes and statistical inference. Ark. Mat. 1 195–277.
  • Hájek, J. (1956). Linear estimation of the mean value of a stationary random process with convex correlation function. Czechoslovak Math. J. 6 94–117.
  • Harman, R. and Štulajter, F. (2010). Optimal prediction designs in finite discrete spectrum linear regression models. Metrika 72 281–294.
  • Kanwal, R. P. (1997). Linear Integral Equations, 2nd ed. Birkhäuser, Boston, MA.
  • Kiefer, J. (1974). General equivalence theory for optimum designs (approximate theory). Ann. Statist. 2 849–879.
  • Kiefer, J. and Wolfowitz, J. (1960). The equivalence of two extremum problems. Canad. J. Math. 12 363–366.
  • Kiseľák, J. and Stehlík, M. (2008). Equidistant and $D$-optimal designs for parameters of Ornstein–Uhlenbeck process. Statist. Probab. Lett. 78 1388–1396.
  • Kruskal, W. (1968). When are Gauss–Markov and least squares estimators identical? A coordinate-free approach. Ann. Math. Statist. 39 70–75.
  • Mason, J. C. and Handscomb, D. C. (2003). Chebyshev Polynomials. Chapman & Hall/CRC, Boca Raton, FL.
  • Mehr, C. B. and McFadden, J. A. (1965). Certain properties of Gaussian processes and their first-passage times. J. R. Stat. Soc. Ser. B Stat. Methodol. 27 505–522.
  • Müller, W. G. and Pázman, A. (2003). Measures for designs in experiments with correlated errors. Biometrika 90 423–434.
  • Näther, W. (1985a). Effective Observation of Random Fields. Teubner-Texte zur Mathematik [Teubner Texts in Mathematics] 72. Teubner, Leipzig.
  • Näther, W. (1985b). Exact design for regression models with correlated errors. Statistics 16 479–484.
  • Pázman, A. and Müller, W. G. (2001). Optimal design of experiments subject to correlated errors. Statist. Probab. Lett. 52 29–34.
  • Pukelsheim, F. (2006). Optimal Design of Experiments. Classics in Applied Mathematics 50. SIAM, Philadelphia, PA. Reprint of the 1993 original.
  • Rao, C. R. (1967). Least squares theory using an estimated dispersion matrix and its application to measurement of signals. In Proc. Fifth Berkeley Sympos. Math. Statist. and Probability (Berkeley, Calif., 1965/66), Vol. I: Statistics 355–372. Univ. California Press, Berkeley, CA.
  • Sacks, J. and Ylvisaker, N. D. (1966). Designs for regression problems with correlated errors. Ann. Math. Statist. 37 66–89.
  • Sacks, J. and Ylvisaker, D. (1968). Designs for regression problems with correlated errors; many parameters. Ann. Math. Statist. 39 49–69.
  • Schmidt, K. M. and Zhigljavsky, A. (2009). A characterization of the arcsine distribution. Statist. Probab. Lett. 79 2451–2455.
  • Torsney, B. (1986). Moment inequalities via optimal design theory. Linear Algebra Appl. 82 237–253.
  • Tsybakov, A. B. (2009). Introduction to Nonparametric Estimation. Springer, New York. Revised and extended from the 2004 French original, Translated by Vladimir Zaiats.
  • Ucinski, D. and Atkinson, A. (2004). Experimental design for time-dependent models with correlated observations. Stud. Nonlinear Dyn. Econom. 8 13.
  • Zhigljavsky, A., Dette, H. and Pepelyshev, A. (2010). A new approach to optimal design for linear models with correlated observations. J. Amer. Statist. Assoc. 105 1093–1103.