The Annals of Applied Probability

Asymptotic optimality of tracking policies in stochastic networks

Nicole Bäuerle

Full-text: Open access


Control problems in stochastic queuing networks are hard to solve. However, it is well known that the fluid model provides a useful approximation to the stochastic network.We will formulate a general class of control problems in stochastic queuing networks and consider the corresponding fluid optimization problem ($F$) which is a deterministic control problem and often easy to solve. Contrary to previous literature, our cost rate function is rather general.The value function of ($F$) provides an asymptotic lower bound on the value function of the stochastic network under fluid scaling. Moreover, we can construct from the optimal control of ($F$) a so-called tracking policy for the stochastic queuing network which achieves the lower bound as the fluid scaling parameter tends to $\infty$. In this case we say that the tracking policy is asymptotically optimal. This statement is true for multiclass queuing networks and admission and routing problems.The convergence is monotone under some convexity assumptions. The tracking policy approach also shows that a given fluid model solution can be attained as a fluid limit of the original discrete model.

Article information

Ann. Appl. Probab. Volume 10, Number 4 (2000), 1065-1083.

First available in Project Euclid: 22 April 2002

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 60K25: Queueing theory [See also 68M20, 90B22]
Secondary: 68M20: Performance evaluation; queueing; scheduling [See also 60K25, 90Bxx]

Stochastic network Markov decision process fluid model weak convergence stochastic orderings


Bäuerle, Nicole. Asymptotic optimality of tracking policies in stochastic networks. Ann. Appl. Probab. 10 (2000), no. 4, 1065--1083. doi:10.1214/aoap/1019487606.

Export citation


  • Alanyali, M. and Hajek, B. (1998). Analysis of simple algorithms for dynamic load balancing. Math. Oper. Res. 22 840-871.
  • Atkins, D. and Chen, H. (1995). Performance evaluation of scheduling control of queueing networks: fluid model heuristics. Queueing Sys. 21 391-413.
  • Avram, F. (1997). Optimal control of fluid limits of queueing networks and stochasticity corrections. In Mathematics of Stochastic Manufacturing Systems. Lectures in Applied Mathematics (G. G. Yin and Q. Zhang, eds.) 33 1-36. Amer. Math. Soc., Providence, RI.
  • Avram, F., Bertsimas, D. and Ricard, M. (1995). Fluid models of sequencing problems in open queueing networks: an optimal control approach. In Stochastic Networks (F. P. Kelly and R. J. Williams, eds.) 199-234. Springer, NewYork.
  • B¨auerle, N. (1998). The advantage of small machines in a stochastic fluid production process. Math. Methods Oper. Res. 47 83-97.
  • B¨auerle, N. and Rieder, U. (2000). Optimal control of single-server fluid networks. Queueing Systems 35 185-200.
  • Bertsekas, D. P. and Shreve, S. E. (1978). Stochastic Optimal Control: The Discrete Time Case. Academic Press, NewYork.
  • Bramson, M. (1996). Convergence to equilibria for fluid models of FIFO queueing networks. Queueing Systems 22 5-45.
  • Chen, H. (1995). Fluid approximations and stability of multiclass queueing networks: workconserving disciplines. Ann. Appl. Probab. 5 637-665.
  • Chen, R. R. and Meyn, S. (1998). Value iteration and optimization of multiclass queueing networks. Queueing Systems 32 65-97.
  • Dai, J. G. (1995). On positive Harris recurrence of multiclass queueing networks: a unified approach via fluid limit models. Ann. Appl. Probab. 5 49-77.
  • Dai, J. G. (1998). Stability of fluid and stochastic processing networks. Lecture note, Univ. Aarhus.
  • Davis, M. H. A. (1993). Markov Models and Optimization. Chapman and Hall, London.
  • Ethier, S. N. and Kurtz, T. G. (1986). Markov Processes. Wiley, NewYork.
  • Harrison, J. M. (1996). The BIGSTEP approach to flowmanagement in stochastic processing networks. In Stochastic Networks: Stochastic Control Theory and Applications. Royal Statistical Society Lecture Notes Series (F. Kelly, S. Zachary and I. Ziedins, eds.) 57-90. Clarendon, Oxford.
  • Kitaev, M. and Rykov, V. (1995). Controlled Queueing Systems. CRC Press, Boca Raton, FL.
  • Kumar, S. and Kumar, P. R. (1996). Fluctuation smoothing policies are stable for stochastic re-entrant lines. J. Discrete Event Dynam. Sys. Theory Appl. 6 361-370.
  • Kushner, H. J. (1990). Numerical methods for stochastic control problems in continuous time. SIAM J. Control Optim. 5 999-1048.
  • Kushner, H. J. and Dupuis, P. G. (1992). Numerical Methods for Stochastic Control Problems in Continuous Time. Springer, NewYork.
  • Luo, X. and Bertsimas, D. (1998). A newalgorithm for state-constrained separated continuous linear programs. SIAM J. Control Optim. 37 177-210.
  • Maglaras, C. (1998). Discrete-reviewpolicies for scheduling stochastic networks: fluid-scale asymptotic optimality. Preprint.
  • Pollard, D. (1984). Convergence of Stochastic Processes. Springer, NewYork.
  • Pullan, M. C. (1995). Forms of optimal solutions for separated continuous linear programs. SIAM J. Control Optim. 33 1952-1977.
  • Rieder, U. (1975). Bayesian dynamic programming. Adv. in Appl. Probab. 7 330-348.
  • Rybko, A. N. and Stolyar, A. L. (1992). Ergodicity of stochastic processes describing the operation of open queueing networks. Problems Inform. Transmission 28 199-220.
  • Seierstad, A. and Sydsæter, K. (1987). Optimal Control Theory with Economic Applications. North-Holland, Amsterdam.
  • Sennott, L. I. (1998). Stochastic Dynamic Programming and the Control of Queues. Wiley, NewYork.
  • Shaked, M. and Shanthikumar, J. G. (1994). Stochastic Orders and Their Applications. Academic, NewYork.
  • Stidham, S. and Weber, R. (1993). A survey of Markov decision models for control of networks of queues. Queueing Systems 13 291-314.
  • Tijms, H. C. (1986). Stochastic Modelling and Analysis: A Computational Approach. Wiley, Chichester.
  • Weiss, G. (1995). On optimal draining of re-entrant fluid lines. In Stochastic Networks (F. P. Kelly and R. J. Williams, eds.) 91-103. Springer, NewYork.
  • Weiss, G. (1996). Optimal draining of fluid re-entrant lines: some solved examples. In Stochastic Networks: Stochastic Control Theory and Applications. Royal Statistical Society Lecture Notes Series (F. Kelly, S. Zachary, and I. Ziedins, eds.) 19-34. Clarendon, Oxford.
  • Weiss, G. (1997). Algorithm for minimum wait draining of two-station fluid re-entrant line. Technical report, Dept. Statistics, Univ. Haifa.
  • Williams, R. J. (1998). Some recent developments for queueing networks. Probability towards 2000. Lecture Notes in Statist. 340-356. Springer, NewYork.