## The Annals of Statistics

### Adaptive optimization and $D$-optimum experimental design

Luc Pronzato

#### Abstract

We consider the situation where one has to maximize a function $\eta(\theta, \mathbf{x})$ with respect to $\mathbf{x} \epsilon \mathbb{R}^q$, when $\theta$ is unknown and estimated by least squares through observations $y_k = \mathbf{f}^{\top(\mathbf{x}_k)\theta + \varepsilon_k$, with $\varepsilon_k$ some random error. Classical applications are regulation and extremum control problems. The approach we adopt corresponds to maximizing the sum of the current estimated objective and a penalization for poor estimation: $\mathbf{x}_{k + 1}$ maximizes $\eta(\hat{\theta}^k, \mathbf{x}) + (\alpha_k/k), d_k(\mathbf{x})$, with $\hat{\theta}^k$ the estimated value of $\theta$ at step $k$ and $d_k$ the penalization function. Sufficient conditions for strong consistency of $\hat{\theta}^k$ and for almost sure convergence of $(1/k) \Sigma_{i=1}^k \eta(\theta, \mathbf{x}_i)$ to the maximum value of $\eta(\theta, \mathbf{x})$ are derived in the case where $d_k(\cdot)$ is the variance function used in the sequential construction of $D$-optimum designs. A classical sequential scheme from adaptive control is shown not to satisfy these conditions, and numerical simulations confirm that it indeed has convergence problems.

#### Article information

Source
Ann. Statist. Volume 28, Number 6 (2000), 1743-1761.

Dates
First available in Project Euclid: 12 March 2002

https://projecteuclid.org/euclid.aos/1015957479

Digital Object Identifier
doi:10.1214/aos/1015957479

Mathematical Reviews number (MathSciNet)
MR1835050

Zentralblatt MATH identifier
1103.93320

Subjects
Primary: 62F12: Asymptotic properties of estimators 62L05: Sequential design

#### Citation

Pronzato, Luc. Adaptive optimization and $D$-optimum experimental design. Ann. Statist. 28 (2000), no. 6, 1743--1761. doi:10.1214/aos/1015957479. https://projecteuclid.org/euclid.aos/1015957479

#### References

• Allison, B. J., Tessier, P. J. C. and Dumont, G. A. (1995). Comparison ofsuboptimal dual adaptive controllers. In Proceedings of the 3rd European Control Conference 36-43. Rome.
• Alster, J. and Blanger, P. R. (1974). A technique for dual adaptive control. Automatica 10 627-634.
• Bar-Shalom, Y. (1981). Stochastic dynamic programming: caution and probing. IEEE Trans. Automatic Control 26 1184-1195.
• Bar-Shalom, Y. and Tse, E. (1974). Dual effect, certainty equivalence, and separation in stochastic control. IEEE Trans. Automatic Control 19 494-500.
• Bozin, A. S. and Zarrop, M. B. (1991). Selftuning optimizer-convergence and robustness properties. In Proceedings of the 1st European Control Conference 672-677. Herm´es, Paris. Fel'dbaum, A. A. (1960a). Dual control theory. I. Avtomat. i Telemek. 21 1240-1249. Fel'dbaum, A. A. (1960b). Dual control theory. II. Avtomat. i Telemek. 21 1453-1464. Fel'dbaum, A. A. (1961a). The theory ofdual control. III. Avtomat. i Telemek. 22 3-16. Fel'dbaum, A. A. (1961b). The theory ofdual control. IV. Avtomat. i Telemek. 22 129-142.
• Ginebra, J. and Clayton, M. K. (1995). Response surface bandits. J. Roy. Statist. Soc. Ser. B 771-784.
• Kulcs´ar, C., Pronzato, L. and Walter, E. (1996). Dual control oflinearly parameterised models via prediction ofposterior densities. European J. Control 2 135-143.
• Lai, T. L. and Wei, C. Z. (1982). Least squares estimates in stochastic regression models with applications to identification and control ofdynamic systems. Ann. Statist. 10 154-166.
• Lai, T. L. and Wei, C. Z. (1987). Asymptotically efficient self-tuning regulators. SIAM J. Control Optim. 25 466-481.
• Ljung, L. (1977). Analysis ofrecursive stochastic algorithms. IEEE Trans. Automat. Control 22 551-575.
• P´azman, A. (1974). A convergence theorem in the theory of D-optimum experimental designs. Ann. Statist. 2 216-218.
• P´azman, A. (1986). Foundations of Optimum Experimental Design. Reidel, Dordrecht.
• Pronzato, L. and Walter, E. (1993). Experimental design for estimating the optimum point in a response surface. Acta Appl. Math. 33 45-68.
• Pukelsheim, F. (1993). Optimal Experimental Design. Wiley, New York.
• Silvey, S. D. (1980). Optimal Design. Chapman & Hall, London.
• Wellstead, P. E. and Zarrop, M. B. (1991). Self-Tuning Sytems. Wiley, Chichester.
• Wittenmark, B. (1975). An active suboptimal controller for systems with stochastic parameters. Automat. Control TheoryAppl. 3 13-19.
• Wittenmark, B. and Elevitch, C. (1985). An adaptive control algorithm with dual features. In Prep. 7th IFAC/IFORS Symposium on Identification and System Parameter Estimation 587-592, York.
• Wu, C.-F. and Wynn, H. P. (1978). The convergence ofgeneral step-length algorithms for regular optimum design criteria. Ann. Statist. 6 1273-1285.
• Wynn, H.P. (1970). The sequential generation of D-optimum experimental designs. Ann. Math. Statist. 41 1655-1664.