## The Annals of Applied Probability

### Ergodic control of multi-class $M/M/N+M$ queues in the Halfin–Whitt regime

#### Abstract

We study a dynamic scheduling problem for a multi-class queueing network with a large pool of statistically identical servers. The arrival processes are Poisson, and service times and patience times are assumed to be exponentially distributed and class dependent. The optimization criterion is the expected long time average (ergodic) of a general (nonlinear) running cost function of the queue lengths. We consider this control problem in the Halfin–Whitt (QED) regime, that is, the number of servers $n$ and the total offered load $\mathbf{r}$ scale like $n\approx\mathbf{r}+\hat{\rho}\sqrt{\mathbf{r}}$ for some constant $\hat{\rho}$. This problem was proposed in [Ann. Appl. Probab. 14 (2004) 1084–1134, Section 5.2].

The optimal solution of this control problem can be approximated by that of the corresponding ergodic diffusion control problem in the limit. We introduce a broad class of ergodic control problems for controlled diffusions, which includes a large class of queueing models in the diffusion approximation, and establish a complete characterization of optimality via the study of the associated HJB equation. We also prove the asymptotic convergence of the values for the multi-class queueing control problem to the value of the associated ergodic diffusion control problem. The proof relies on an approximation method by spatial truncation for the ergodic control of diffusion processes, where the Markov policies follow a fixed priority policy outside a fixed compact set.

#### Article information

Source
Ann. Appl. Probab., Volume 25, Number 6 (2015), 3511-3570.

Dates
Revised: November 2014
First available in Project Euclid: 1 October 2015

https://projecteuclid.org/euclid.aoap/1443703781

Digital Object Identifier
doi:10.1214/14-AAP1081

Mathematical Reviews number (MathSciNet)
MR3404643

Zentralblatt MATH identifier
1330.60108

#### Citation

Arapostathis, Ari; Biswas, Anup; Pang, Guodong. Ergodic control of multi-class $M/M/N+M$ queues in the Halfin–Whitt regime. Ann. Appl. Probab. 25 (2015), no. 6, 3511--3570. doi:10.1214/14-AAP1081. https://projecteuclid.org/euclid.aoap/1443703781

#### References

• [1] Arapostathis, A., Borkar, V. S. and Ghosh, M. K. (2012). Ergodic Control of Diffusion Processes. Encyclopedia of Mathematics and Its Applications 143. Cambridge Univ. Press, Cambridge.
• [2] Arisawa, M. and Lions, P.-L. (1998). On ergodic stochastic control. Comm. Partial Differential Equations 23 2187–2217.
• [3] Atar, R. (2005). Scheduling control for queueing systems with many servers: Asymptotic optimality in heavy traffic. Ann. Appl. Probab. 15 2606–2650.
• [4] Atar, R. and Biswas, A. (2014). Control of the multiclass $G/G/1$ queue in the moderate deviation regime. Ann. Appl. Probab. 24 2033–2069.
• [5] Atar, R., Giat, C. and Shimkin, N. (2010). The $c\mu/\theta$ rule for many-server queues with abandonment. Oper. Res. 58 1427–1439.
• [6] Atar, R., Giat, C. and Shimkin, N. (2011). On the asymptotic optimality of the $c\mu/\theta$ rule under ergodic cost. Queueing Syst. 67 127–144.
• [7] Atar, R., Mandelbaum, A. and Reiman, M. I. (2004). Scheduling a multi class queue with many exponential servers: Asymptotic optimality in heavy traffic. Ann. Appl. Probab. 14 1084–1134.
• [8] Bogachev, V. I., Krylov, N. V. and Röckner, M. (2001). On regularity of transition probabilities and invariant measures of singular diffusions under minimal conditions. Comm. Partial Differential Equations 26 2037–2080.
• [9] Borkar, V. S. (1989). Optimal Control of Diffusion Processes. Pitman Research Notes in Mathematics Series 203. Longman, Harlow.
• [10] Brown, L., Gans, N., Mandelbaum, A., Sakov, A., Shen, H., Zeltyn, S. and Zhao, L. (2005). Statistical analysis of a telephone call center: A queueing-science perspective. J. Amer. Statist. Assoc. 100 36–50.
• [11] Budhiraja, A., Ghosh, A. and Liu, X. (2014). Scheduling control for Markov-modulated single-server multiclass queueing systems in heavy traffic. Queueing Syst. 78 57–97.
• [12] Budhiraja, A., Ghosh, A. P. and Lee, C. (2011). Ergodic rate control problem for single class queueing networks. SIAM J. Control Optim. 49 1570–1606.
• [13] Dai, J. G. and Tezcan, T. (2008). Optimal control of parallel server systems with many servers in heavy traffic. Queueing Syst. 59 95–134.
• [14] Dieker, A. B. and Gao, X. (2013). Positive recurrence of piecewise Ornstein–Uhlenbeck processes and common quadratic Lyapunov functions. Ann. Appl. Probab. 23 1291–1317.
• [15] Gamarnik, D. and Stolyar, A. L. (2012). Multiclass multiserver queueing system in the Halfin-Whitt heavy traffic regime: Asymptotics of the stationary distribution. Queueing Syst. 71 25–51.
• [16] Gamarnik, D. and Zeevi, A. (2006). Validity of heavy traffic steady-state approximation in generalized Jackson networks. Ann. Appl. Probab. 16 56–90.
• [17] Garnett, O., Mandelbaum, A. and Reiman, M. I. (2002). Designing a call center with impatient customers. Manuf. Serv. Oper. Manag. 4 208–227.
• [18] Gilbarg, D. and Trudinger, N. S. (1983). Elliptic Partial Differential Equations of Second Order, 2nd ed. Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences] 224. Springer, Berlin.
• [19] Gurvich, I. (2014). Diffusion models and steady-state approximations for exponentially ergodic Markovian queues. Ann. Appl. Probab. 24 2527–2559.
• [20] Gyöngy, I. and Krylov, N. (1996). Existence of strong solutions for Itô’s stochastic equations via approximations. Probab. Theory Related Fields 105 143–158.
• [21] Halfin, S. and Whitt, W. (1981). Heavy-traffic limits for queues with many exponential servers. Oper. Res. 29 567–588.
• [22] Harrison, J. M. and Zeevi, A. (2004). Dynamic scheduling of a multiclass queue in the Halfin–Whitt heavy traffic regime. Oper. Res. 52 243–257.
• [23] Ichihara, N. and Sheu, S.-J. (2013). Large time behavior of solutions of Hamilton–Jacobi–Bellman equations with quadratic nonlinearity in gradients. SIAM J. Math. Anal. 45 279–306.
• [24] Kallenberg, O. (2002). Foundations of Modern Probability, 2nd ed. Springer, New York.
• [25] Kim, J. and Ward, A. R. (2013). Dynamic scheduling of a $GI/GI/1+GI$ queue with multiple customer classes. Queueing Syst. 75 339–384.
• [26] Koçağa, Y. L. and Ward, A. R. (2010). Admission control for a multi-server queue with abandonment. Queueing Syst. 65 275–323.
• [27] Krylov, N. V. (1980). Controlled Diffusion Processes. Applications of Mathematics 14. Springer, New York. Translated from the Russian by A. B. Aries.
• [28] Mandelbaum, A. and Stolyar, A. L. (2004). Scheduling flexible servers with convex delay costs: Heavy-traffic optimality of the generalized $c\mu$-rule. Oper. Res. 52 836–855.
• [29] Ocone, D. and Weerasinghe, A. (2003). Degenerate variance control in the one-dimensional stationary case. Electron. J. Probab. 8 no. 24, 27 pp. (electronic).
• [30] Pang, G., Talreja, R. and Whitt, W. (2007). Martingale proofs of many-server heavy-traffic limits for Markovian queues. Probab. Surv. 4 193–267.
• [31] Stannat, W. (1999). (Nonsymmetric) Dirichlet operators on $L^{1}$: Existence, uniqueness and associated Markov processes. Ann. Scuola Norm. Sup. Pisa Cl. Sci. (4) 28 99–140.
• [32] van Mieghem, J. A. (1995). Dynamic scheduling with convex delay costs: The generalized $c\mu$ rule. Ann. Appl. Probab. 5 809–833.
• [33] Yosida, K. (1980). Functional Analysis, 6th ed. Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences] 123. Springer, Berlin.