The Annals of Applied Probability

Diffusion approximations for controlled stochastic networks: An asymptotic bound for the value function

Amarjit Budhiraja and Arka Prasanna Ghosh

Full-text: Open access


We consider the scheduling control problem for a family of unitary networks under heavy traffic, with general interarrival and service times, probabilistic routing and infinite horizon discounted linear holding cost. A natural nonanticipativity condition for admissibility of control policies is introduced. The condition is seen to hold for a broad class of problems. Using this formulation of admissible controls and a time-transformation technique, we establish that the infimum of the cost for the network control problem over all admissible sequencing control policies is asymptotically bounded below by the value function of an associated diffusion control problem (the Brownian control problem). This result provides a useful bound on the best achievable performance for any admissible control policy for a wide class of networks.

Article information

Ann. Appl. Probab., Volume 16, Number 4 (2006), 1962-2006.

First available in Project Euclid: 17 January 2007

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 60K25: Queueing theory [See also 68M20, 90B22] 68M20: Performance evaluation; queueing; scheduling [See also 60K25, 90Bxx] 90B22: Queues and service [See also 60K25, 68M20] 90B35: Scheduling theory, deterministic [See also 68M20]
Secondary: 60J70: Applications of Brownian motions and diffusion theory (population genetics, absorption problems, etc.) [See also 92Dxx]

Control of queuing networks heavy traffic Brownian control problem equivalent workload formulation unitary networks asymptotic optimality


Budhiraja, Amarjit; Ghosh, Arka Prasanna. Diffusion approximations for controlled stochastic networks: An asymptotic bound for the value function. Ann. Appl. Probab. 16 (2006), no. 4, 1962--2006. doi:10.1214/105051606000000457.

Export citation


  • Ata, B. and Kumar, S. (2005). Heavy traffic analysis of open processing networks with complete resource pooling: Asymptotic optimality of discrete review policies. Ann. Appl. Probab. 15 331–391.
  • Bell, S. L. and Williams, R. J. (2001). Dynamic scheduling of a system with two parallel servers in heavy traffic with resource pooling: Asymptotic optimality of a threshold policy. Ann. Appl. Probab. 11 608–664.
  • Billingsley, P. (1999). Convergence of Probability Measures, 2nd ed. Wiley, New York.
  • Bohm, V. (1975). On the continuity of the optimal policy set of linear programs. SIAM J. Appl. Math. 28 303–306.
  • Bramson, M. and Williams, R. J. (2003). Two workload properties for Brownian networks. Queueing Systems 45 191–221.
  • Bramson, M. and Williams, R. J. (2000). On dynamic scheduling of stochastic networks in heavy traffic and some new results for the workload process. In Proceedings of the 39th IEEE Conference on Decision and Control 516–521.
  • Budhiraja, A. and Ghosh, A. P. (2005). A large deviations approach to asymptotically optimal control of crisscross network in heavy traffic. Ann. Appl. Probab. 15 1887–1935.
  • Dai, J. G. and Williams, R. J. (1995). Existence and uniqueness of semimartingale reflecting Brownian motions in convex polyhedrons. Theory Probab. Appl. 40 1–40.
  • Durrett, R. (1996). Probability: Theory and Examples, 2nd ed. Duxbury Press, Belmont, CA.
  • Ethier, S. N. and Kurtz, T. G. (1986). Markov Processes, Characterization and Convergence. Wiley, New York.
  • Harrison, J. M. and Meighem, J. A. (1997). Dynamic control of Brownian networks: State space collapse and equivalent workload formulation. Ann. Appl. Probab. 7 747–771.
  • Harrison, J. M. (1988). Brownian models of queueing networks with heterogeneous customer population. In Stochastic Differential Systems, Stochastic Control Theory and Their Applications 147–186. Springer, New York.
  • Harrison, J. M. (2000). Brownian models of open processing networks: Canonical representation of workload. Ann. Appl. Probab. 10 75–103. [Correction 13 (2003) 390–393].
  • Karatzas, I. and Shreve, S. E. (1991). Brownian Motion and Stochastic Calculus, 2nd ed. Springer, New York.
  • Kurtz, T. G. (1991). Random time changes and convergence in distribution under the Meyer–Zheng conditions. Ann. Probab. 19 1010–1034.
  • Kurtz, T. G. and Stockbridge, R. H. (1999). Martingale problems and linear programs for singular control. In 37th Annual Allerton Conference on Communication Control and Computing (Monticello, Ill.) 11–20. Univ. Illinois, Urbana-Champaign.
  • Kushner, H. J. and Martins, L. F. (1996). Heavy traffic analysis of a controlled multiclass queuing network via weak convergence methods. SIAM J. Control Optim. 34 1781–1797.
  • Kushner, H. J. and Ramachandran, K. M. (1989). Optimal and approximately optimal control policies for queues in heavy traffic. SIAM J. Control Optim. 27 1293–1318.
  • Meyer, P. A. and Zheng, W. A. (1984). Tightness criteria for laws of semimartingales. Ann. Inst. H. Poincaré Probab. Statist. 20 353–372.
  • Meyn, S. P. (2003). Sequencing and routing in multiclass queueing networks. Part II: Workload relaxations. SIAM J. Control Optim. 42 178–217.
  • Protter, P. E. (2004). Stochastic Integration and Differential Equations, 2nd ed. Springer, Berlin.
  • Thorisson, H. (2000). Coupling, Stationarity, and Regeneration. Springer, New York.