Annals of Statistics

Optimum design accounting for the global nonlinear behavior of the model

Andrej Pázman and Luc Pronzato

Full-text: Open access


Among the major difficulties that one may encounter when estimating parameters in a nonlinear regression model are the nonuniqueness of the estimator, its instability with respect to small perturbations of the observations and the presence of local optimizers of the estimation criterion.

We show that these estimability issues can be taken into account at the design stage, through the definition of suitable design criteria. Extensions of $E$-, $c$- and $G$-optimality criteria are considered, which when evaluated at a given $\theta^{0}$ (local optimal design), account for the behavior of the model response $\eta(\theta )$ for $\theta $ far from $\theta^{0}$. In particular, they ensure some protection against close-to-overlapping situations where $\|\eta(\theta )-\eta(\theta^{0})\|$ is small for some $\theta $ far from $\theta^{0}$. These extended criteria are concave and necessary and sufficient conditions for optimality (equivalence theorems) can be formulated. They are not differentiable, but when the design space is finite and the set $\Theta$ of admissible $\theta $ is discretized, optimal design forms a linear programming problem which can be solved directly or via relaxation when $\Theta$ is just compact. Several examples are presented.

Article information

Ann. Statist., Volume 42, Number 4 (2014), 1426-1451.

First available in Project Euclid: 25 June 2014

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62K05: Optimal designs
Secondary: 62J02: General nonlinear regression

Optimal design nonlinear least-squares estimability curvature


Pázman, Andrej; Pronzato, Luc. Optimum design accounting for the global nonlinear behavior of the model. Ann. Statist. 42 (2014), no. 4, 1426--1451. doi:10.1214/14-AOS1232.

Export citation


  • Atkinson, A. C., Chaloner, K., Herzberg, A. M. and Juritz, J. (1993). Optimal experimental designs for properties of a compartmental model. Biometrics 49 325–337.
  • Bates, D. M. and Watts, D. G. (1980). Relative curvature measures of nonlinearity. J. R. Stat. Soc. Ser. B Stat. Methodol. 42 1–25.
  • Bonnans, J. F., Gilbert, J. C., Lemaréchal, C. and Sagastizábal, C. A. (2006). Numerical Optimization: Theoretical and Practical Aspects, 2nd ed. Springer, Berlin.
  • Chavent, G. (1983). Local stability of the output least square parameter estimation technique. Mat. Apl. Comput. 2 3–22.
  • Chavent, G. (1990). A new sufficient condition for the well-posedness of nonlinear least square problems arising in identification and control. In Analysis and Optimization of Systems (A. Bensoussan and J. L. Lions, eds.). Lecture Notes in Control and Inform. Sci. 144 452–463. Springer, Berlin.
  • Chavent, G. (1991). New size$\times$curvature conditions for strict quasiconvexity of sets. SIAM J. Control Optim. 29 1348–1372.
  • Clyde, M. and Chaloner, K. (2002). Constrained design strategies for improving normal approximations in nonlinear regression problems. J. Statist. Plann. Inference 104 175–196.
  • Dem’yanov, V. F. and Malozemov, V. N. (1974). Introduction to Minimax. Dover, New York.
  • Demidenko, E. Z. (1989). Optimizatsiya i Regressiya. Nauka, Moscow.
  • Demidenko, E. (2000). Is this the least squares estimate? Biometrika 87 437–452.
  • Fedorov, V. V. (1972). Theory of Optimal Experiments. Academic Press, New York.
  • Fedorov, V. V. and Hackl, P. (1997). Model-Oriented Design of Experiments. Lecture Notes in Statistics 125. Springer, New York.
  • Fedorov, V. V. and Leonov, S. L. (2014). Optimal Design for Nonlinear Response Models. CRC Press, Boca Raton, FL.
  • Gauchi, J.-P. and Pázman, A. (2006). Designs in nonlinear regression by stochastic minimization of functionals of the mean square error matrix. J. Statist. Plann. Inference 136 1135–1152.
  • Hamilton, D. C. and Watts, D. G. (1985). A quadratic design criterion for precise estimation in nonlinear regression models. Technometrics 27 241–250.
  • Harville, D. A. (1997). Matrix Algebra from a Statistician’s Perspective. Springer, Heidelberg.
  • Kelley, J. E. Jr. (1960). The cutting-plane method for solving convex programs. J. Soc. Indust. Appl. Math. 8 703–712.
  • Kiefer, J. and Wolfowitz, J. (1960). The equivalence of two extremum problems. Canad. J. Math. 12 363–366.
  • Kieffer, M. and Walter, E. (1998). Interval analysis for guaranteed nonlinear parameter estimation. In MODA 5—Advances in Model-Oriented Data Analysis and Experimental Design (Marseilles, 1998) (A. C. Atkinson, L. Pronzato and H. P. Wynn, eds.) 115–125. Physica, Heidelberg.
  • Lemaréchal, C., Nemirovskii, A. and Nesterov, Y. (1995). New variants of bundle methods. Math. Program. 69 111–147.
  • Nesterov, Y. (2004). Introductory Lectures on Convex Optimization: A Basic Course. Applied Optimization 87. Kluwer Academic, Boston, MA.
  • Pázman, A. (1984). Nonlinear least squares—uniqueness versus ambiguity. Math. Operationsforsch. Statist. Ser. Statist. 15 323–336.
  • Pázman, A. (1993). Nonlinear Statistical Models. Kluwer, Dordrecht.
  • Pázman, A. and Pronzato, L. (1992). Nonlinear experimental design based on the distribution of estimators. J. Statist. Plann. Inference 33 385–402.
  • Pronzato, L. (2009). On the regularization of singular $c$-optimal designs. Math. Slovaca 59 611–626.
  • Pronzato, L., Huang, C. Y. and Walter, E. (1991). Nonsequential ${T}$-optimal design for model discrimination: New algorithms. In Proc. PROBASTAT’91 (A. Pázman and J. Volaufová, eds.) 130–136. Mathematical Institute of the Slovak Academy of Sciences, Bratislava.
  • Pronzato, L. and Pázman, A. (1994). Second-order approximation of the entropy in nonlinear least-squares estimation. Kybernetika (Prague) 30 187–198.
  • Pronzato, L. and Pázman, A. (2013). Design of Experiments in Nonlinear Models: Asymptotic Normality, Optimality Criteria and Small-Sample Properties. Lecture Notes in Statistics 212. Springer, New York.
  • Pukelsheim, F. (1993). Optimal Experimental Design. Wiley, New York.
  • Ratkowsky, D. A. (1983). Nonlinear Regression Modelling. Dekker, New York.
  • Shimizu, K. and Aiyoshi, E. (1980). Necessary conditions for min-max problems and algorithms by a relaxation procedure. IEEE Trans. Automat. Control 25 62–66.
  • Silvey, S. D. (1980). Optimal Design. Chapman & Hall, London.
  • Tang, B. (1993). Orthogonal array-based Latin hypercubes. J. Amer. Statist. Assoc. 88 1392–1397.
  • Walter, E., ed. (1987). Identifiability of Parametric Models. Pergamon Press, Oxford.
  • Walter, E. and Pronzato, L. (1995). Identifiabilities and nonlinearities. In Nonlinear Systems, Vol. 1 (A. J. Fossard and D. Normand-Cyrot, eds.) 111–143. Chapman & Hall, London.