Source: Ann. Appl. Probab.
Volume 15, Number 2
A system manager dynamically controls a diffusion process Z that lives in a finite interval [0,b]. Control takes the form of a negative drift rate θ that is chosen from a fixed set A of available values. The controlled process evolves according to the differential relationship dZ=dX−θ(Z) dt+dL−dU, where X is a (0,σ) Brownian motion, and L and U are increasing processes that enforce a lower reflecting barrier at Z=0 and an upper reflecting barrier at Z=b, respectively. The cumulative cost process increases according to the differential relationship dξ=c(θ(Z)) dt+p dU, where c(⋅) is a nondecreasing cost of control and p>0 is a penalty rate associated with displacement at the upper boundary. The objective is to minimize long-run average cost. This problem is solved explicitly, which allows one to also solve the following, essentially equivalent formulation: minimize the long-run average cost of control subject to an upper bound constraint on the average rate at which U increases. The two special problem features that allow an explicit solution are the use of a long-run average cost criterion, as opposed to a discounted cost criterion, and the lack of state-related costs other than boundary displacement penalties. The application of this theory to power control in wireless communication is discussed.
Altman, E. (1999). Constrained Markov Decision Processes. Chapman and Hall/CRC, London.
Ata, B. (2003). Dynamic control of stochastic networks. Ph.D. dissertation, Graduate School of Business, Stanford Univ. Available at http://www.kellogg.nwu.edu/faculty/ata/htm/Research/Publications.htm.
Benes, V. and Karatzas, I. (1981). Optimal stationary linear control of the Wiener process. J. Optim. Theory Appl. 35 611--633.
Mathematical Reviews (MathSciNet): MR663333
Benes, V. E., Shepp, L. A. and Witsenhausen, H. (1980). Some solvable stochastic control problems. Stochastics 4 39--83.
Mathematical Reviews (MathSciNet): MR587428
Berry, R. (2000). Power and delay trade-offs in fading channels. Ph.D. dissertation, MIT.
Berry, R. and Gallager, R. (2002). Communication over fading channels with delay constraints. IEEE Trans. Inform. Theory 48 1135--1149.
Fleming, W. and Rishel, R. (1975). Deterministic and Stochastic Optimal Control. Springer, New York.
Mathematical Reviews (MathSciNet): MR454768
George, J. M. and Harrison, J. M. (2001). Dynamic control of a queue with adjustable service rate. Oper. Res. 49 720--731.
Harrison, J. M. (1985). Brownian Motion and Stochastic Flow Systems. Wiley, New York.
Mathematical Reviews (MathSciNet): MR798279
Karatzas, I. and Shreve, S. (1991). Brownian Motion and Stochastic Calculus, 2nd ed. Springer, New York.
Kushner, H. (1971). Introduction to Stochastic Control Theory. Holt, Rinehart and Winston, New York.
Mathematical Reviews (MathSciNet): MR280248
Kushner, H. J. (2001). Heavy Traffic Analysis of Controlled Queueing and Communication Networks. Springer, Berlin.
Kushner, H. J. and Dupuis, P. (1992). Numerical Methods for Stochastic Control Problems in Continuous Time. Springer, New York.
Plambeck, E., Kumar, S. and Harrison, J. M. (2001). A multiclass queue in heavy traffic with throughput time constraints: Asymptotically optimal dynamic controls. Queueing Systems 39 23--54.
Uysal-Biyikoglu, E., Prabhakar, B. and El Gamal, A. (2002). Energy-efficient packet transmission over a wireless link. IEEE/ACM Transactions on Networking 10 487--499.