The Annals of Applied Probability

Ergodic control of multi-class $M/M/N+M$ queues in the Halfin–Whitt regime

Ari Arapostathis, Anup Biswas, and Guodong Pang

Full-text: Open access


We study a dynamic scheduling problem for a multi-class queueing network with a large pool of statistically identical servers. The arrival processes are Poisson, and service times and patience times are assumed to be exponentially distributed and class dependent. The optimization criterion is the expected long time average (ergodic) of a general (nonlinear) running cost function of the queue lengths. We consider this control problem in the Halfin–Whitt (QED) regime, that is, the number of servers $n$ and the total offered load $\mathbf{r}$ scale like $n\approx\mathbf{r}+\hat{\rho}\sqrt{\mathbf{r}}$ for some constant $\hat{\rho}$. This problem was proposed in [Ann. Appl. Probab. 14 (2004) 1084–1134, Section 5.2].

The optimal solution of this control problem can be approximated by that of the corresponding ergodic diffusion control problem in the limit. We introduce a broad class of ergodic control problems for controlled diffusions, which includes a large class of queueing models in the diffusion approximation, and establish a complete characterization of optimality via the study of the associated HJB equation. We also prove the asymptotic convergence of the values for the multi-class queueing control problem to the value of the associated ergodic diffusion control problem. The proof relies on an approximation method by spatial truncation for the ergodic control of diffusion processes, where the Markov policies follow a fixed priority policy outside a fixed compact set.

Article information

Ann. Appl. Probab., Volume 25, Number 6 (2015), 3511-3570.

Received: April 2014
Revised: November 2014
First available in Project Euclid: 1 October 2015

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 60K25: Queueing theory [See also 68M20, 90B22]
Secondary: 68M20: Performance evaluation; queueing; scheduling [See also 60K25, 90Bxx] 90B22: Queues and service [See also 60K25, 68M20] 90B36: Scheduling theory, stochastic [See also 68M20]

Multi-class Markovian queues reneging/abandonment Halfin–Whitt (QED) regime diffusion scaling long time-average control ergodic control stable Markov optimal control spatial truncation asymptotic optimality


Arapostathis, Ari; Biswas, Anup; Pang, Guodong. Ergodic control of multi-class $M/M/N+M$ queues in the Halfin–Whitt regime. Ann. Appl. Probab. 25 (2015), no. 6, 3511--3570. doi:10.1214/14-AAP1081.

Export citation


  • [1] Arapostathis, A., Borkar, V. S. and Ghosh, M. K. (2012). Ergodic Control of Diffusion Processes. Encyclopedia of Mathematics and Its Applications 143. Cambridge Univ. Press, Cambridge.
  • [2] Arisawa, M. and Lions, P.-L. (1998). On ergodic stochastic control. Comm. Partial Differential Equations 23 2187–2217.
  • [3] Atar, R. (2005). Scheduling control for queueing systems with many servers: Asymptotic optimality in heavy traffic. Ann. Appl. Probab. 15 2606–2650.
  • [4] Atar, R. and Biswas, A. (2014). Control of the multiclass $G/G/1$ queue in the moderate deviation regime. Ann. Appl. Probab. 24 2033–2069.
  • [5] Atar, R., Giat, C. and Shimkin, N. (2010). The $c\mu/\theta$ rule for many-server queues with abandonment. Oper. Res. 58 1427–1439.
  • [6] Atar, R., Giat, C. and Shimkin, N. (2011). On the asymptotic optimality of the $c\mu/\theta$ rule under ergodic cost. Queueing Syst. 67 127–144.
  • [7] Atar, R., Mandelbaum, A. and Reiman, M. I. (2004). Scheduling a multi class queue with many exponential servers: Asymptotic optimality in heavy traffic. Ann. Appl. Probab. 14 1084–1134.
  • [8] Bogachev, V. I., Krylov, N. V. and Röckner, M. (2001). On regularity of transition probabilities and invariant measures of singular diffusions under minimal conditions. Comm. Partial Differential Equations 26 2037–2080.
  • [9] Borkar, V. S. (1989). Optimal Control of Diffusion Processes. Pitman Research Notes in Mathematics Series 203. Longman, Harlow.
  • [10] Brown, L., Gans, N., Mandelbaum, A., Sakov, A., Shen, H., Zeltyn, S. and Zhao, L. (2005). Statistical analysis of a telephone call center: A queueing-science perspective. J. Amer. Statist. Assoc. 100 36–50.
  • [11] Budhiraja, A., Ghosh, A. and Liu, X. (2014). Scheduling control for Markov-modulated single-server multiclass queueing systems in heavy traffic. Queueing Syst. 78 57–97.
  • [12] Budhiraja, A., Ghosh, A. P. and Lee, C. (2011). Ergodic rate control problem for single class queueing networks. SIAM J. Control Optim. 49 1570–1606.
  • [13] Dai, J. G. and Tezcan, T. (2008). Optimal control of parallel server systems with many servers in heavy traffic. Queueing Syst. 59 95–134.
  • [14] Dieker, A. B. and Gao, X. (2013). Positive recurrence of piecewise Ornstein–Uhlenbeck processes and common quadratic Lyapunov functions. Ann. Appl. Probab. 23 1291–1317.
  • [15] Gamarnik, D. and Stolyar, A. L. (2012). Multiclass multiserver queueing system in the Halfin-Whitt heavy traffic regime: Asymptotics of the stationary distribution. Queueing Syst. 71 25–51.
  • [16] Gamarnik, D. and Zeevi, A. (2006). Validity of heavy traffic steady-state approximation in generalized Jackson networks. Ann. Appl. Probab. 16 56–90.
  • [17] Garnett, O., Mandelbaum, A. and Reiman, M. I. (2002). Designing a call center with impatient customers. Manuf. Serv. Oper. Manag. 4 208–227.
  • [18] Gilbarg, D. and Trudinger, N. S. (1983). Elliptic Partial Differential Equations of Second Order, 2nd ed. Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences] 224. Springer, Berlin.
  • [19] Gurvich, I. (2014). Diffusion models and steady-state approximations for exponentially ergodic Markovian queues. Ann. Appl. Probab. 24 2527–2559.
  • [20] Gyöngy, I. and Krylov, N. (1996). Existence of strong solutions for Itô’s stochastic equations via approximations. Probab. Theory Related Fields 105 143–158.
  • [21] Halfin, S. and Whitt, W. (1981). Heavy-traffic limits for queues with many exponential servers. Oper. Res. 29 567–588.
  • [22] Harrison, J. M. and Zeevi, A. (2004). Dynamic scheduling of a multiclass queue in the Halfin–Whitt heavy traffic regime. Oper. Res. 52 243–257.
  • [23] Ichihara, N. and Sheu, S.-J. (2013). Large time behavior of solutions of Hamilton–Jacobi–Bellman equations with quadratic nonlinearity in gradients. SIAM J. Math. Anal. 45 279–306.
  • [24] Kallenberg, O. (2002). Foundations of Modern Probability, 2nd ed. Springer, New York.
  • [25] Kim, J. and Ward, A. R. (2013). Dynamic scheduling of a $GI/GI/1+GI$ queue with multiple customer classes. Queueing Syst. 75 339–384.
  • [26] Koçağa, Y. L. and Ward, A. R. (2010). Admission control for a multi-server queue with abandonment. Queueing Syst. 65 275–323.
  • [27] Krylov, N. V. (1980). Controlled Diffusion Processes. Applications of Mathematics 14. Springer, New York. Translated from the Russian by A. B. Aries.
  • [28] Mandelbaum, A. and Stolyar, A. L. (2004). Scheduling flexible servers with convex delay costs: Heavy-traffic optimality of the generalized $c\mu$-rule. Oper. Res. 52 836–855.
  • [29] Ocone, D. and Weerasinghe, A. (2003). Degenerate variance control in the one-dimensional stationary case. Electron. J. Probab. 8 no. 24, 27 pp. (electronic).
  • [30] Pang, G., Talreja, R. and Whitt, W. (2007). Martingale proofs of many-server heavy-traffic limits for Markovian queues. Probab. Surv. 4 193–267.
  • [31] Stannat, W. (1999). (Nonsymmetric) Dirichlet operators on $L^{1}$: Existence, uniqueness and associated Markov processes. Ann. Scuola Norm. Sup. Pisa Cl. Sci. (4) 28 99–140.
  • [32] van Mieghem, J. A. (1995). Dynamic scheduling with convex delay costs: The generalized $c\mu$ rule. Ann. Appl. Probab. 5 809–833.
  • [33] Yosida, K. (1980). Functional Analysis, 6th ed. Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences] 123. Springer, Berlin.