The Annals of Statistics

Monotonic convergence of a general algorithm for computing optimal designs

Yaming Yu

Full-text: Open access


Monotonic convergence is established for a general class of multiplicative algorithms introduced by Silvey, Titterington and Torsney [Comm. Statist. Theory Methods 14 (1978) 1379–1389] for computing optimal designs. A conjecture of Titterington [Appl. Stat. 27 (1978) 227–234] is confirmed as a consequence. Optimal designs for logistic regression are used as an illustration.

Article information

Ann. Statist. Volume 38, Number 3 (2010), 1593-1606.

First available: 24 March 2010

Permanent link to this document

Digital Object Identifier

Zentralblatt MATH identifier

Mathematical Reviews number (MathSciNet)

Primary: 62K05: Optimal designs

A-optimality auxiliary variables c-optimality D-optimality experimental design generalized linear models multiplicative algorithm


Yu, Yaming. Monotonic convergence of a general algorithm for computing optimal designs. The Annals of Statistics 38 (2010), no. 3, 1593--1606. doi:10.1214/09-AOS761.

Export citation


  • Atwood, C. L. (1973). Sequences converging to D-optimal designs of experiments. Ann. Statist. 1 342–352.
  • Chaloner, K. and Larntz, K. (1989). Optimal Bayesian design applied to logistic regression experiments. J. Statist. Plann. Inference 21 191–208.
  • Chaloner, K. and Verdinelli, I. (1995). Bayesian experimental design: A review. Statist. Sci. 10 273–304.
  • Chernoff, H. (1953). Locally optimal design for estimating parameters. Ann. Math. Statist. 24 586–602.
  • Csiszár, I. and Tusnady, G. (1984). Information geometry and alternating minimization procedures. Statist. Decisions (Suppl. 1) 205–237.
  • Dempster, A. P., Laird, N. M. and Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm (with discussion). J. Roy. Statist. Soc. Ser. B 39 1–38.
  • Dette, H., Pepelyshev, A. and Zhigljavsky, A. (2008). Improving updating rules in multiplicative algorithms for computing D-optimal designs. Comput. Statist. Data Anal. 53 312–320.
  • Fedorov, V. V. (1972). Theory of Optimal Experiments. Academic Press, New York.
  • Fellman, J. (1974). On the allocation of linear observations (Thesis). Soc. Sci. Fenn. Comment. Phys.-Math. 44 27–78.
  • Fellman, J. (1989). An empirical study of a class of iterative searches for optimal designs. J. Statist. Plann. Inference 21 85–92.
  • Harman, R. and Pronzato, L. (2007). Improvements on removing nonoptimal support points in D-optimum design algorithms. Statist. Probab. Lett. 77 90–94.
  • Harman, R. and Trnovská, M. (2009). Approximate D-optimal designs of experiments on the convex hull of a finite set of information matrices. Math. Slovaca 59 693–704.
  • Kiefer, J. (1974). General equivalence theory for optimum designs (approximate theory). Ann. Statist. 2 849–879.
  • Kiefer, J. and Wolfowitz, J. (1960). The equivalence of two extremum problems. Canad. J. Math. 12 363–366.
  • Li, G. and Majumdar, D. (2008). D-optimal designs for logistic models with three and four parameters. J. Statist. Plann. Inference 138 1950–1959.
  • Meng, X.-L. and van Dyk, D. (1997). The EM algorithm—an old folk-song sung to a fast new tune (with discussion). J. Roy. Statist. Soc. Ser. B 59 511–567.
  • Pázman, A. (1986). Foundations of Optimum Experimental Design. Reidel, Dordrecht.
  • Pronzato, L., Wynn, H. and Zhigljavsky, A. (2000). Dynamical Search: Applications of Dynamical Systems in Search and Optimization. Chapman & Hall/CRC, Boca Raton.
  • Pukelsheim, F. (1993). Optimal Design of Experiments. Wiley, New York.
  • Pukelsheim, F. and Torsney, B. (1991). Optimal weights for experimental designs on linearly independent support points. Ann. Statist. 19 1614–1625.
  • Silvey, S. D. (1980). Optimal Design. Chapman & Hall, London.
  • Silvey, S. D., Titterington, D. M. and Torsney, B. (1978). An algorithm for optimal designs on a finite design space. Comm. Statist. Theory Methods 14 1379–1389.
  • Titterington, D. M. (1976). Algorithms for computing D-optimal design on finite design spaces. In Proc. of the 1976 Conf. on Information Science and Systems 3 213–216. John Hopkins Univ.
  • Titterington, D. M. (1978). Estimation of correlation coefficients by ellipsoidal trimming. J. Appl. Stat. 27 227–234.
  • Torsney, B. (1983). A moment inequality and monotonicity of an algorithm. In Proceedings of the International Symposium on Semi-Infinite Programming and Appl. (K. O. Kortanek and A. V. Fiacco, eds.). Lecture Notes in Economics and Mathematical Systems 215 249–260. Springer, Berlin.
  • Torsney, B. (2007). W-iterations and ripples therefrom. In Optimal Design and Related Areas in Optimization and Statistics (L. Pronzato and A. Zhigljavsky, eds.) 1–12. Springer, New York.
  • Torsney, B. and Mandal, S. (2006). Two classes of multiplicative algorithms for constructing optimizing distributions. Comput. Statist. Data Anal. 51 1591–1601.
  • Torsney, B. and Martín-Martín, R. (2009). Multiplicative algorithms for computing optimum designs. J. Statist. Plann. Inference 139 3947–3961.
  • Whittle, P. (1973). Some general points in the theory of optimal experimental design. J. Roy. Statist. Soc. Ser. B 35 123–130.
  • Wu, C. F. and Wynn, H. P. (1978). The convergence of general step-length algorithms for regular optimum design criteria. Ann. Statist. 6 1273–1285.
  • Wynn, H. P. (1972). Results in the theory and construction of D-optimum experimental designs. J. Roy. Statist. Soc. Ser. B 34 133–147.
  • Yu, Y. (2008). A bit of information theory, and the data augmentation algorithm converges. IEEE Trans. Inform. Theory 54 5186–5188.
  • Yu, Y. (2009). Monotonic convergence of a general algorithm for computing optimal designs (extended version). Technical report. Available at arXiv:0905.2646.