## The Annals of Applied Probability

### Asymptotic optimality of tracking policies in stochastic networks

Nicole Bäuerle

#### Abstract

Control problems in stochastic queuing networks are hard to solve. However, it is well known that the fluid model provides a useful approximation to the stochastic network.We will formulate a general class of control problems in stochastic queuing networks and consider the corresponding fluid optimization problem ($F$) which is a deterministic control problem and often easy to solve. Contrary to previous literature, our cost rate function is rather general.The value function of ($F$) provides an asymptotic lower bound on the value function of the stochastic network under fluid scaling. Moreover, we can construct from the optimal control of ($F$) a so-called tracking policy for the stochastic queuing network which achieves the lower bound as the fluid scaling parameter tends to $\infty$. In this case we say that the tracking policy is asymptotically optimal. This statement is true for multiclass queuing networks and admission and routing problems.The convergence is monotone under some convexity assumptions. The tracking policy approach also shows that a given fluid model solution can be attained as a fluid limit of the original discrete model.

#### Article information

Source
Ann. Appl. Probab. Volume 10, Number 4 (2000), 1065-1083.

Dates
First available in Project Euclid: 22 April 2002

http://projecteuclid.org/euclid.aoap/1019487606

Digital Object Identifier
doi:10.1214/aoap/1019487606

Mathematical Reviews number (MathSciNet)
MR1810864

Zentralblatt MATH identifier
1057.90003

#### Citation

Bäuerle, Nicole. Asymptotic optimality of tracking policies in stochastic networks. Ann. Appl. Probab. 10 (2000), no. 4, 1065--1083. doi:10.1214/aoap/1019487606. http://projecteuclid.org/euclid.aoap/1019487606.

#### References

• Alanyali, M. and Hajek, B. (1998). Analysis of simple algorithms for dynamic load balancing. Math. Oper. Res. 22 840-871.
• Atkins, D. and Chen, H. (1995). Performance evaluation of scheduling control of queueing networks: fluid model heuristics. Queueing Sys. 21 391-413.
• Avram, F. (1997). Optimal control of fluid limits of queueing networks and stochasticity corrections. In Mathematics of Stochastic Manufacturing Systems. Lectures in Applied Mathematics (G. G. Yin and Q. Zhang, eds.) 33 1-36. Amer. Math. Soc., Providence, RI.
• Avram, F., Bertsimas, D. and Ricard, M. (1995). Fluid models of sequencing problems in open queueing networks: an optimal control approach. In Stochastic Networks (F. P. Kelly and R. J. Williams, eds.) 199-234. Springer, NewYork.
• B¨auerle, N. (1998). The advantage of small machines in a stochastic fluid production process. Math. Methods Oper. Res. 47 83-97.
• B¨auerle, N. and Rieder, U. (2000). Optimal control of single-server fluid networks. Queueing Systems 35 185-200.
• Bertsekas, D. P. and Shreve, S. E. (1978). Stochastic Optimal Control: The Discrete Time Case. Academic Press, NewYork.
• Bramson, M. (1996). Convergence to equilibria for fluid models of FIFO queueing networks. Queueing Systems 22 5-45.
• Chen, H. (1995). Fluid approximations and stability of multiclass queueing networks: workconserving disciplines. Ann. Appl. Probab. 5 637-665.
• Chen, R. R. and Meyn, S. (1998). Value iteration and optimization of multiclass queueing networks. Queueing Systems 32 65-97.
• Dai, J. G. (1995). On positive Harris recurrence of multiclass queueing networks: a unified approach via fluid limit models. Ann. Appl. Probab. 5 49-77.
• Dai, J. G. (1998). Stability of fluid and stochastic processing networks. Lecture note, Univ. Aarhus.
• Davis, M. H. A. (1993). Markov Models and Optimization. Chapman and Hall, London.
• Ethier, S. N. and Kurtz, T. G. (1986). Markov Processes. Wiley, NewYork.
• Harrison, J. M. (1996). The BIGSTEP approach to flowmanagement in stochastic processing networks. In Stochastic Networks: Stochastic Control Theory and Applications. Royal Statistical Society Lecture Notes Series (F. Kelly, S. Zachary and I. Ziedins, eds.) 57-90. Clarendon, Oxford.
• Kitaev, M. and Rykov, V. (1995). Controlled Queueing Systems. CRC Press, Boca Raton, FL.
• Kumar, S. and Kumar, P. R. (1996). Fluctuation smoothing policies are stable for stochastic re-entrant lines. J. Discrete Event Dynam. Sys. Theory Appl. 6 361-370.
• Kushner, H. J. (1990). Numerical methods for stochastic control problems in continuous time. SIAM J. Control Optim. 5 999-1048.
• Kushner, H. J. and Dupuis, P. G. (1992). Numerical Methods for Stochastic Control Problems in Continuous Time. Springer, NewYork.
• Luo, X. and Bertsimas, D. (1998). A newalgorithm for state-constrained separated continuous linear programs. SIAM J. Control Optim. 37 177-210.
• Maglaras, C. (1998). Discrete-reviewpolicies for scheduling stochastic networks: fluid-scale asymptotic optimality. Preprint.
• Pollard, D. (1984). Convergence of Stochastic Processes. Springer, NewYork.
• Pullan, M. C. (1995). Forms of optimal solutions for separated continuous linear programs. SIAM J. Control Optim. 33 1952-1977.
• Rieder, U. (1975). Bayesian dynamic programming. Adv. in Appl. Probab. 7 330-348.
• Rybko, A. N. and Stolyar, A. L. (1992). Ergodicity of stochastic processes describing the operation of open queueing networks. Problems Inform. Transmission 28 199-220.
• Seierstad, A. and Sydsæter, K. (1987). Optimal Control Theory with Economic Applications. North-Holland, Amsterdam.
• Sennott, L. I. (1998). Stochastic Dynamic Programming and the Control of Queues. Wiley, NewYork.
• Shaked, M. and Shanthikumar, J. G. (1994). Stochastic Orders and Their Applications. Academic, NewYork.
• Stidham, S. and Weber, R. (1993). A survey of Markov decision models for control of networks of queues. Queueing Systems 13 291-314.
• Tijms, H. C. (1986). Stochastic Modelling and Analysis: A Computational Approach. Wiley, Chichester.
• Weiss, G. (1995). On optimal draining of re-entrant fluid lines. In Stochastic Networks (F. P. Kelly and R. J. Williams, eds.) 91-103. Springer, NewYork.
• Weiss, G. (1996). Optimal draining of fluid re-entrant lines: some solved examples. In Stochastic Networks: Stochastic Control Theory and Applications. Royal Statistical Society Lecture Notes Series (F. Kelly, S. Zachary, and I. Ziedins, eds.) 19-34. Clarendon, Oxford.
• Weiss, G. (1997). Algorithm for minimum wait draining of two-station fluid re-entrant line. Technical report, Dept. Statistics, Univ. Haifa.
• Williams, R. J. (1998). Some recent developments for queueing networks. Probability towards 2000. Lecture Notes in Statist. 340-356. Springer, NewYork.