The Annals of Statistics

Computing exact $D$-optimal designs by mixed integer second-order cone programming

Abstract

Let the design of an experiment be represented by an $s$-dimensional vector $\mathbf{w}$ of weights with nonnegative components. Let the quality of $\mathbf{w}$ for the estimation of the parameters of the statistical model be measured by the criterion of $D$-optimality, defined as the $m$th root of the determinant of the information matrix $M(\mathbf{w} )=\sum_{i=1}^{s}w_{i}A_{i}A_{i}^{T}$, where $A_{i},i=1,\ldots,s$ are known matrices with $m$ rows.

In this paper, we show that the criterion of $D$-optimality is second-order cone representable. As a result, the method of second-order cone programming can be used to compute an approximate $D$-optimal design with any system of linear constraints on the vector of weights. More importantly, the proposed characterization allows us to compute an exact $D$-optimal design, which is possible thanks to high-quality branch-and-cut solvers specialized to solve mixed integer second-order cone programming problems. Our results extend to the case of the criterion of $D_{K}$-optimality, which measures the quality of $\mathbf{w}$ for the estimation of a linear parameter subsystem defined by a full-rank coefficient matrix $K$.

We prove that some other widely used criteria are also second-order cone representable, for instance, the criteria of $A$-, $A_{K}$-, $G$- and $I$-optimality.

We present several numerical examples demonstrating the efficiency and general applicability of the proposed method. We show that in many cases the mixed integer second-order cone programming approach allows us to find a provably optimal exact design, while the standard heuristics systematically miss the optimum.

Article information

Source
Ann. Statist., Volume 43, Number 5 (2015), 2198-2224.

Dates
Revised: January 2015
First available in Project Euclid: 16 September 2015

https://projecteuclid.org/euclid.aos/1442364150

Digital Object Identifier
doi:10.1214/15-AOS1339

Mathematical Reviews number (MathSciNet)
MR3396983

Zentralblatt MATH identifier
1331.62384

Subjects
Primary: 62K05: Optimal designs

Citation

Sagnol, Guillaume; Harman, Radoslav. Computing exact $D$-optimal designs by mixed integer second-order cone programming. Ann. Statist. 43 (2015), no. 5, 2198--2224. doi:10.1214/15-AOS1339. https://projecteuclid.org/euclid.aos/1442364150

References

• [1] Alizadeh, F. and Goldfarb, D. (2003). Second-order cone programming. Math. Program. 95 3–51.
• [2] Andersen, E. D., Jensen, B., Jensen, J., Sandvik, R. and Worsøe, U. (2009). MOSEK Version 6. Technical Report TR–2009–3, MOSEK.
• [3] Atkinson, A. C. and Donev, A. N. (1992). Optimum Experimental Designs 8. Oxford Univ. Press, Oxford.
• [4] Bailey, R. A. and Cameron, P. J. (2009). Combinatorics of optimal designs. In Surveys in Combinatorics 2009. London Mathematical Society Lecture Note Series 365 19–73. Cambridge Univ. Press, Cambridge.
• [5] Ben-Tal, A. and Nemirovski, A. (1987). Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications 2. SIAM, Philadelphia.
• [6] Boyd, S. and Vandenberghe, L. (2004). Convex Optimization. Cambridge Univ. Press, Cambridge.
• [7] Chaloner, K. and Verdinelli, I. (1995). Bayesian experimental design: A review. Statist. Sci. 10 273–304.
• [8] Chen, A. and Esfahanian, A. H. (2005). A demography of $t$-optimal $(n,m)$ graphs where $n<=12$. In AMCS 121–127. CSREA Press, Las Vegas, NV.
• [9] Cook, D. and Fedorov, V. (1995). Constrained optimization of experimental design. Statistics 26 129–178.
• [10] Cook, R. D. and Thibodeau, L. A. (1980). Marginally restricted $D$-optimal designs. J. Amer. Statist. Assoc. 75 366–371.
• [11] Duarte, B. P. M. and Wong, W. K. (2015). Finding Bayesian optimal designs for nonlinear models: A semidefinite programming-based approach. Int. Stat. Rev. To appear. DOI:10.1111/insr.12073.
• [12] Fedorov, V. and Lee, J. (2000). Design of experiments in statistics. In Handbook of Semidefinite Programming (H. Wolkowicz, R. Saigal and L. Vandenberghe, eds.). Kluwer, Dordrecht.
• [13] Fedorov, V. V. (1972). Theory of Optimal Experiments. Academic Press, New York.
• [14] Filová, L., Trnovská, M. and Harman, R. (2012). Computing maximin efficient experimental designs using the methods of semidefinite programming. Metrika 75 709–719.
• [15] Haines, L. M. (1987). The application of the annealing algorithm to the construction of exact optimal designs for linear-regression models. Technometrics 29 439–447.
• [16] Harman, R. (2014). Multiplicative methods for computing $D$-optimal stratified designs of experiments. J. Statist. Plann. Inference 146 82–94.
• [17] Harman, R. and Filová, L. (2014). Computing efficient exact designs of experiments using integer quadratic programming. Comput. Statist. Data Anal. 71 1159–1167.
• [18] Harman, R. and Jurík, T. (2008). Computing $c$-optimal experimental designs using the simplex method of linear programming. Comput. Statist. Data Anal. 53 247–254.
• [19] Harman, R. and Pronzato, L. (2007). Improvements on removing nonoptimal support points in $D$-optimum design algorithms. Statist. Probab. Lett. 77 90–94.
• [20] Heredia-Langner, A., Carlyle, W. M., Montgomery, D. C., Borror, C. M. and Runger, G. C. (2003). Genetic algorithms for the construction of $D$-optimal designs. J. Qual. Technol. 35 28–46.
• [21] IBM ILOG (2009). IBM ILOG CPLEX V12.1. User’s manual for CPLEX. Technical report, International Business Machines Corporation.
• [22] Kiefer, J. and Wolfowitz, J. (1960). The equivalence of two extremum problems. Canad. J. Math. 12 363–366.
• [23] Lobo, M. S., Vandenberghe, L., Boyd, S. and Lebret, H. (1998). Applications of second-order cone programming. Linear Algebra Appl. 284 193–228.
• [24] Löfberg, J. (2004). YALMIP: A toolbox for modeling and optimization in Matlab. In 2004 IEEE International Symposium on Computer Aided Control Systems Design 284–289. IEEE, New York.
• [25] Lu, Z. and Pong, T. K. (2013). Computing optimal experimental designs via interior point method. SIAM J. Matrix Anal. Appl. 34 1556–1580.
• [26] Martín-Martín, R., Torsney, B. and López-Fidalgo, J. (2007). Construction of marginally and conditionally restricted designs using multiplicative algorithms. Comput. Statist. Data Anal. 51 5547–5561.
• [27] Mitchell, T. J. (1974). An algorithm for the construction of “$D$-optimal” experimental designs. Technometrics 16 203–210.
• [28] Papp, D. (2012). Optimal designs for rational function regression. J. Amer. Statist. Assoc. 107 400–411.
• [29] Petingi, L., Boesch, F. and Suffel, C. (1998). On the characterization of graphs with maximum number of spanning trees. Discrete Math. 179 155–166.
• [30] Pronzato, L. and Zhigljavsky, A. (2014). Algorithmic construction of optimal designs on compact sets for concave and differentiable criteria. J. Statist. Plann. Inference 154 141–155.
• [31] Pukelsheim, F. (1993). Optimal Design of Experiments. Wiley, New York.
• [32] Pukelsheim, F. and Rieder, S. (1992). Efficient rounding of approximate designs. Biometrika 79 763–770.
• [33] Rodriguez, M., Jones, B., Borror, C. M. and Montgomery, D. C. (2010). Generating and assessing exact $G$-optimal designs. J. Qual. Technol. 42 3–20.
• [34] Sagnol, G. (2011). Computing optimal designs of multiresponse experiments reduces to second-order cone programming. J. Statist. Plann. Inference 141 1684–1708.
• [35] Sagnol, G. (2012). PICOS, a Python interface to conic optimization solvers. Technical Report No. 12–48, ZIB, http://picos.zib.de.
• [36] Sagnol, G. (2013). On the semidefinite representation of real functions applied to symmetric matrices. Linear Algebra Appl. 439 2829–2843.
• [37] Sagnol, G. and Harman, R. (2013). Computing exact $D$-optimal designs by mixed integer second order cone programming. Preprint. Available at arXiv:1307.4953v2.
• [38] Seber, G. A. F. (2008). A Matrix Handbook for Statisticians. Wiley, Hoboken, NJ.
• [39] Silvey, S. D., Titterington, D. M. and Torsney, B. (1978). An algorithm for optimal designs on a finite design space. Comm. Statist. Theory Methods 7 1379–1389.
• [40] Tack, L. and Vandebroek, M. (2004). Budget constrained run orders in optimum design. J. Statist. Plann. Inference 124 231–249.
• [41] Titterington, D. M. (1976). Algorithms for computing $D$-optimal design on finite design spaces. In Proceedings of the 1976 Conf. on Information Science and Systems 213–216. Dept. of Electronic Engineering, John Hopkins Univ., Baltimore, MD.
• [42] Vandenberghe, L., Boyd, S. and Wu, S.-P. (1998). Determinant maximization with linear matrix inequality constraints. SIAM J. Matrix Anal. Appl. 19 499–533.
• [43] Welch, W. J. (1982). Branch-and-bound search for experimental designs based on $D$-optimality and other criteria. Technometrics 24 41–48.
• [44] Wright, S. E., Sigal, B. M. and Bailer, A. J. (2010). Workweek optimization of experimental designs: Exact designs for variable sampling costs. J. Agric. Biol. Environ. Stat. 15 491–509.
• [45] Wynn, H. P. (1970). The sequential generation of $D$-optimum experimental designs. Ann. Math. Statist. 41 1655–1664.
• [46] Yang, M., Biedermann, S. and Tang, E. (2013). On optimal designs for nonlinear models: A general and efficient algorithm. J. Amer. Statist. Assoc. 108 1411–1420.
• [47] Yu, Y. (2010). Monotonic convergence of a general algorithm for computing optimal designs. Ann. Statist. 38 1593–1606.
• [48] Yu, Y. (2011). $D$-optimal designs via a cocktail algorithm. Stat. Comput. 21 475–481.