Some Stochastic Approximation Procedures for Use in Process Control

Abstract

In this paper we shall consider mathematical models motivated by chemical process control problems in which we wish to control the process to hold some property of the output material as nearly constant as possible. For example, we might control the viscosity of the process output by varying the setting of a valve in the cooling water line to a heat exchanger. We shall suppose that the viscosity is observed at equally spaced times and that these observations form a sequence $Y_1, Y_2, Y_3, \cdots$. The control procedure specifies the valve setting $X_1$ to be used during the time interval $(0, 1\rbrack$. Then, immediately after observing $Y_n$, we reset the cooling water valve to the position $X_{n+1}$ in accordance with the control procedure. The value of $X_{n+1}$ may depend on the prior values $X_1, \cdots, X_n$ and $Y_1, \cdots, Y_n$. In choosing $X_{n+1}$ we hope to hold $Y_{n+1}$ as close to a fixed value $Y_0$ as possible. More specifically, our objective will be to keep values of the loss $E(Y_n - Y_0)^2$ small. In this paper it is assumed that time lags attributable to process dynamics are negligible when compared with the interval between observations. Parts of the theory are being generalized to allow for the presence of such time lags and no serious difficulties have as yet been encountered in this type of generalization. The basic process and the general control procedure are described in mathematical terms in Sections 2 and 3. The concept of a system consisting of the process and the control procedure is then introduced in Section 4. The specific type of control procedure we shall consider is a generalization of the simple proportional control procedure discussed by Box and Jenkins [1]. This procedure requires that we specify $X_1$ and then recursively specify $X_{n+1} = X_n - a_n(Y_n - Y_0)$. We may place bounds on $X_n$ as in (8) below. The sequence $\{a_n\}$ is a sequence of positive numbers which may be specified a priori as in Section 5 or sequentially as in Section 9. In Sections 6 and 7 we study the performance of certain processes when the sequence $\{a_n\}$ is specified a priori. Most important are cases where $a_n$ converges to a small value $a$ near zero. With the class of processes used in this paper, it turns out that this procedure for choosing $X_n$ is quite similar to the stochastic approximation procedure first presented by Robbins and Monro in 1951 [7] and later generalized by Dvoretzky [5] and other authors. A comprehensive review of work on stochastic approximation is contained in a recent article by Schmetterer [8]. In Section 8 we specialize to the study of certain stationary processes. Attention is focused on the asymptotic performance of the controlled process as a function of the limit $a$ of the sequence $\{a_n\}$ in our control procedure. Of particular interest is the asymptotic loss given by $\lim_{n\rightarrow\infty} E(Y_n - Y_0)^2$. A procedure for the sequential choice of the sequence $\{a_n\}$ is proposed in Section 9 and its theoretical properties are studied in Section 10. Conditions are given under which both $a_n$ and $E(Y_n - Y_0)^2$ converge. It turns out that this procedure for the sequential choice of $a_n$ is also a stochastic approximation procedure of the Robbins-Monro type. Numerical examples given in Section 11 indicate that the asymptotic loss obtained by using this procedure for sequentially choosing $a_n$ may be nearly as small as the asymptotic loss obtained when the sequence $\{a_n\}$ is chosen a priori to converge to the value of $a$ that minimizes the asymptotic loss. However, when $a_n$ is chosen sequentially, we need much less a priori knowledge of the process.