The Annals of Applied Probability

Rate control under heavy traffic with strategic servers

Erhan Bayraktar, Amarjit Budhiraja, and Asaf Cohen

Full-text: Access denied (no subscription detected)

We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text


We consider a large queueing system that consists of many strategic servers that are weakly interacting. Each server processes jobs from its unique critically loaded buffer and controls the rate of arrivals and departures associated with its queue to minimize its expected cost. The rates and the cost functions in addition to depending on the control action, can depend, in a symmetric fashion, on the size of the individual queue and the empirical measure of the states of all queues in the system. In order to determine an approximate Nash equilibrium for this finite player game, we construct a Lasry–Lions-type mean-field game (MFG) for certain reflected diffusions that governs the limiting behavior. Under conditions, we establish the convergence of the Nash-equilibrium value for the finite size queuing system to the value of the MFG.

Article information

Ann. Appl. Probab., Volume 29, Number 1 (2019), 1-35.

Received: February 2017
Revised: August 2017
First available in Project Euclid: 5 December 2018

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: Heavy traffic limits queuing systems strategic servers mean-field games rate control reflected diffusions

60K25 91A13 60K35 93E20 60H30 60F17


Bayraktar, Erhan; Budhiraja, Amarjit; Cohen, Asaf. Rate control under heavy traffic with strategic servers. Ann. Appl. Probab. 29 (2019), no. 1, 1--35. doi:10.1214/17-AAP1349.

Export citation


  • [1] Achdou, Y., Camilli, F. and Capuzzo-Dolcetta, I. (2013). Mean field games: Convergence of a finite difference method. SIAM J. Numer. Anal. 51 2585–2612.
  • [2] Achdou, Y. and Capuzzo-Dolcetta, I. (2010). Mean field games: Numerical methods. SIAM J. Numer. Anal. 48 1136–1162.
  • [3] Achdou, Y. and Porretta, A. (2016). Convergence of a finite difference scheme to weak solutions of the system of partial differential equations arising in mean field games. SIAM J. Numer. Anal. 54 161–186.
  • [4] Aliprantis, C. D. and Border, K. C. (2006). Infinite Dimensional Analysis: A Hitchhiker’s Guide, 3rd ed. Springer, Berlin.
  • [5] Antunes, N., Fricker, C., Robert, P. and Tibi, D. (2008). Stochastic networks with multiple stable points. Ann. Probab. 36 255–278.
  • [6] Ata, B., Harrison, J. M. and Shepp, L. A. (2005). Drift rate control of a Brownian processing system. Ann. Appl. Probab. 15 1145–1160.
  • [7] Baccelli, F., Karpelevich, F., Kelbert, M. Y., Puhalskii, A., Rybko, A. and Suhov, Y. M. (1992). A mean-field limit for a class of queueing networks. J. Stat. Phys. 66 803–825.
  • [8] Bayraktar, E., Budhiraja, A. and Cohen, A. (2017). A numerical scheme for a mean field game in some queuing systems based on Markov chain approximation method. SIAM J. Control Optim. To appear.
  • [9] Bayraktar, E., Horst, U. and Sircar, R. (2006). A limit theorem for financial markets with inert investors. Math. Oper. Res. 31 789–810.
  • [10] Bayraktar, E., Horst, U. and Sircar, R. (2007). Queuing theoretic approaches to financial price fluctuations. Handbooks Oper. Res. Management Sci. 15 637–677.
  • [11] Bayraktar, E. and Ludkovski, M. (2011). Optimal trade execution in illiquid markets. Math. Finance 21 681–701.
  • [12] Bayraktar, E. and Ludkovski, M. (2014). Liquidation in limit order books with controlled intensity. Math. Finance 24 627–650.
  • [13] Bayraktar, E. and Zhang, Y. (2016). A rank-based mean field game in the strong formulation. Electron. Commun. Probab. 21 Paper No. 72.
  • [14] Benaim, M. and Le Boudec, J.-Y. (2008). A class of mean field interaction models for computer and communication systems. Perform. Eval. 65 823–838.
  • [15] Benamou, J.-D. and Carlier, G. (2015). Augmented Lagrangian methods for transport optimization, mean field games and degenerate elliptic equations. J. Optim. Theory Appl. 167 1–26.
  • [16] Billingsley, P. (1999). Convergence of Probability Measures, 2nd ed. Wiley, New York.
  • [17] Blair, J., Johnson, P. and Duck, P. (2015). Analysis of optimal liquidation in limit order books. Preprint. Available at
  • [18] Bobbio, A., Gribaudo, M. and Telek, M. (2008). Analysis of large scale interacting systems by mean field method. In Fifth International Conference on Quantitative Evaluation of Systems (QEST’08) 215–224. IEEE, New York.
  • [19] Borkar, V. S. (1989). Optimal Control of Diffusion Processes. Pitman Research Notes in Mathematics Series 203. Longman Scientific & Technical, Harlow.
  • [20] Borovkov, K. A. (1998). Propagation of chaos for queueing networks. Theory Probab. Appl. 42 385–394.
  • [21] Bowe, M., Hyde, S. and Johnson, I. (2009). Determining the intensity of buy and sell limit order submissions: A look at the market preopening period.
  • [22] Budhiraja, A. and Friedlander, E. (2018). Diffusion approximations for controlled weakly interacting large finite state systems with simultaneous jumps. Ann. Appl. Probab. 28 204–249.
  • [23] Budhiraja, A., Ghosh, A. P. and Lee, C. (2011). Ergodic rate control problem for single class queueing networks. SIAM J. Control Optim. 49 1570–1606.
  • [24] Cardaliaguet, P. (2013). Notes on mean field games. Available at
  • [25] Carlini, E. and Silva, F. J. (2014). A fully discrete semi-Lagrangian scheme for a first order mean field game problem. SIAM J. Numer. Anal. 52 45–67.
  • [26] Carlini, E. and Silva, F. J. (2015). A semi-Lagrangian scheme for a degenerate second order mean field game system. Discrete Contin. Dyn. Syst. 35 4269–4292.
  • [27] Carmona, R. and Delarue, F. (2013). Probabilistic analysis of mean-field games. SIAM J. Control Optim. 51 2705–2734.
  • [28] Carmona, R. and Lacker, D. (2015). A probabilistic weak formulation of mean field games and applications. Ann. Appl. Probab. 25 1189–1231.
  • [29] Chassagneux, J.-F., Crisan, D. and Delarue, F. (2017). Numerical method for FBSDEs of McKean–Vlasov type. Available at arXiv:1703.02007 [math.PR].
  • [30] Chen, H. and Yao, D. D. (2001). Fundamentals of Queuing Networks: Performance, Asymptotics, and Optimization. Springer, Berlin.
  • [31] Dai, J. G. and Villiams, R. J. (1995). Existence and uniqueness of semimartingale reflecting Brownian motions in convex polyhedra. Theory Probab. Appl. 40 1–40.
  • [32] El Karoui, N., Hu̇u̇ Nguyen, D. and Jeanblanc-Picqué, M. (1987). Compactification methods in the control of degenerate diffusions: Existence of an optimal control. Stochastics 20 169–219.
  • [33] Fendick, K. W. and Rodrigues, M. A. (1994). Asymptotic analysis of adaptive rate control for diverse sources with delayed feedback. IEEE Trans. Inform. Theory 40 2008–2025.
  • [34] Fischer, M. (2017). On the connection between symmetric $n$-player games and mean field games. Ann. Appl. Probab. 27 757–810.
  • [35] Fleming, W. H. (1976). Generalized solutions in optimal stochastic control. Technical report, DTIC document.
  • [36] Föllmer, H. and Horst, U. (2001). Convergence of locally and globally interacting Markov chains. Stochastic Process. Appl. 96 99–121.
  • [37] Gibbens, R. J., Hunt, P. J. and Kelly, F. P. (1990). Bistability in communication networks. In Disorder in Physical Systems 113–127. Oxford Univ. Press, New York.
  • [38] Gomes, D. A., Mohr, J. and Souza, R. R. (2013). Continuous time finite state mean field games. Appl. Math. Optim. 68 99–143.
  • [39] Gopalakrishnan, R., Doroudi, S., Ward, A. R. and Wierman, A. (2017). Routing and staffing when servers are strategic. Oper. Res. 64 1033–1050.
  • [40] Graham, C. (2000). Chaoticity on path space for a queueing network with selection of the shortest queue among several. J. Appl. Probab. 37 198–211.
  • [41] Graham, C. and Robert, P. (2009). Interacting multi-class transmissions in large stochastic networks. Ann. Appl. Probab. 19 2334–2361.
  • [42] Guéant, O. (2012). New numerical methods for mean field games with quadratic costs. Netw. Heterog. Media 7 315–336.
  • [43] Guéant, O., Lasry, J.-M. and Lions, P.-L. (2011). Mean field games and applications. In Paris–Princeton Lectures on Mathematical Finance 2010. Lecture Notes in Math. 2003 205–266. Springer, Berlin.
  • [44] Guéant, O. and Lehalle, C.-A. (2015). General intensity shapes in optimal liquidation. Math. Finance 25 457–495.
  • [45] Hu, Y. and Peng, S. (1997). A stability theorem of backward stochastic differential equations and its application. C. R. Acad. Sci. Paris Sér. I Math. 324 1059–1064.
  • [46] Huang, M., Caines, P. E. and Malhamé, R. P. (2007). The Nash certainty equivalence principle and Mckean–Vlasov systems: An invariance principle and entry adaptation. In 46th IEEE Conference on Decision and Control 121–126. IEEE, New York.
  • [47] Huang, M., Malhamé, R. P. and Caines, P. E. (2006). Large population stochastic dynamic games: Closed-loop McKean–Vlasov systems and the Nash certainty equivalence principle. Commun. Inf. Syst. 6 221–251.
  • [48] Hunt, P. J. and Kurtz, T. G. (1994). Large loss networks. Stochastic Process. Appl. 53 363–378.
  • [49] Joffe, A. and Métivier, M. (1986). Weak convergence of sequences of semimartingales with applications to multitype branching processes. Adv. in Appl. Probab. 20–65.
  • [50] Kotelenez, P. M. and Kurtz, T. G. (2010). Macroscopic limits for stochastic partial differential equations of McKean–Vlasov type. Probab. Theory Related Fields 146 189–222.
  • [51] Kruk, L., Lehoczky, J., Ramanan, K. and Shreve, S. (2007). An explicit formula for the Skorokhod map on $[0,a]$. Ann. Probab. 35 1740–1768.
  • [52] Kushner, H. J. (2001). Heavy Traffic Analysis of Controlled Queueing and Communication Networks. In Stochastic Modelling and Applied Probability. Applications of Mathematics (New York) 47. Springer, New York.
  • [53] Kushner, H. J. and Dupuis, P. G. (1992). Numerical Methods for Stochastic Control Problems in Continuous Time. Applications of Mathematics (New York) 24. Springer, New York.
  • [54] Lachapelle, A., Lasry, J., Lehalle, C. and Lions, P. (2015). Efficiency of the price formation process in presence of high frequency participants: A mean field game analysis. Available at
  • [55] Lachapelle, A., Salomon, J. and Turinici, G. (2010). Computation of mean field equilibria in economics. Math. Models Methods Appl. Sci. 20 567–588.
  • [56] Lacker, D. (2015). A general characterization of the mean field limit for stochastic differential games. Probab. Theory Related Fields 1–68.
  • [57] Lacker, D. (2015). Mean field games via controlled martingale problems: Existence of Markovian equilibria. Stochastic Process. Appl. 125 2856–2894.
  • [58] Lasry, J.-M. and Lions, P.-L. (2006). Jeux à champ moyen. I. Le cas stationnaire. C. R. Math. Acad. Sci. Paris 343 619–625.
  • [59] Lasry, J.-M. and Lions, P.-L. (2006). Jeux à champ moyen. II. Horizon fini et contrôle optimal. C. R. Math. Acad. Sci. Paris 343 679–684.
  • [60] Lasry, J.-M. and Lions, P.-L. (2007). Mean field games. Jpn. J. Math. 2 229–260.
  • [61] Li, J., Bhattacharyya, R., Paul, S., Shakkottai, S. and Subramanian, V. (2015). Incentivizing sharing in realtime d2d streaming networks: A mean field game perspective. In IEEE Conference on Computer Communications (INFOCOM 2015) 2119–2127. IEEE, New York.
  • [62] Lieberman, G. M. (1996). Second Order Parabolic Differential Equations. World Scientific, River Edge, NJ.
  • [63] Manjrekar, M., Ramaswamy, V. and Shakkottai, S. (2014). A mean field game approach to scheduling in cellular systems. In INFOCOM, 2014 Proceedings IEEE 1554–1562. IEEE, New York.
  • [64] Schauder, J. P. (1930). Der fixpunktsatz in funktionalräumen. Studia Math. 2 171–180.
  • [65] Smart, D. R. (1974). Fixed Point Theorems. Cambridge Tracts in Mathematics 66. Cambridge Univ. Press, London.
  • [66] Sznitman, A.-S. (1991). Topics in propagation of chaos. In École D’Été de Probabilités de Saint-Flour XIX—1989. Lecture Notes in Math. 1464 165–251. Springer, Berlin.
  • [67] Tsybakov, A. B. (2009). Introduction to Nonparametric Estimation. Springer, New York. Revised and extended from the 2004 French original. Translated by Vladimir Zaiats.
  • [68] Villani, C. (2009). Optimal Transport. Old and New. Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences] 338. Springer, Berlin.
  • [69] Vvedenskaya, N. D. and Suhov, Y. M. (1997). Dobrushin’s mean-field approximation for a queue with dynamic routing. Research Report RR-3328, Projet MEVAL, INRIA.
  • [70] Wiecek, P., Altman, E. and Ghosh, A. (2016). Mean-field game approach to admission control of an $M/M/\infty$ queue with shared service cost. Dyn. Games Appl. 6 538–566.