The Annals of Applied Probability

Diffusion models and steady-state approximations for exponentially ergodic Markovian queues

Itai Gurvich

Full-text: Open access


Motivated by queues with many servers, we study Brownian steady-state approximations for continuous time Markov chains (CTMCs). Our approximations are based on diffusion models (rather than a diffusion limit) whose steady-state, we prove, approximates that of the Markov chain with notable precision. Strong approximations provide such “limitless” approximations for process dynamics. Our focus here is on steady-state distributions, and the diffusion model that we propose is tractable relative to strong approximations.

Within an asymptotic framework, in which a scale parameter $n$ is taken large, a uniform (in the scale parameter) Lyapunov condition imposed on the sequence of diffusion models guarantees that the gap between the steady-state moments of the diffusion and those of the properly centered and scaled CTMCs shrinks at a rate of $\sqrt{n}$.

Our proofs build on gradient estimates for solutions of the Poisson equations associated with the (sequence of) diffusion models and on elementary martingale arguments. As a by-product of our analysis, we explore connections between Lyapunov functions for the fluid model, the diffusion model and the CTMC.

Article information

Ann. Appl. Probab., Volume 24, Number 6 (2014), 2527-2559.

First available in Project Euclid: 26 August 2014

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 60K25: Queueing theory [See also 68M20, 90B22] 90B20: Traffic problems 90B36: Scheduling theory, stochastic [See also 68M20] 49L20: Dynamic programming method 60F17: Functional limit theorems; invariance principles

Markovian queues steady-state many servers heavy-traffic Halfin–Whitt regime steady state approximations strong approximations for queues


Gurvich, Itai. Diffusion models and steady-state approximations for exponentially ergodic Markovian queues. Ann. Appl. Probab. 24 (2014), no. 6, 2527--2559. doi:10.1214/13-AAP984.

Export citation


  • [1] Ata, B. and Gurvich, I. (2012). On optimality gaps in the Halfin–Whitt regime. Ann. Appl. Probab. 22 407–455.
  • [2] Baxendale, P. H. (2005). Renewal theory and computable convergence rates for geometrically ergodic Markov chains. Ann. Appl. Probab. 15 700–738.
  • [3] Borkar, V. and Budhiraja, A. (2004/05). Ergodic control for constrained diffusions: Characterization using HJB equations. SIAM J. Control Optim. 43 1467–1492.
  • [4] Brémaud, P. (1981). Point Processes and Queues: Martingale Dynamics. Springer, New York.
  • [5] Budhiraja, A. and Lee, C. (2007). Long time asymptotics for constrained diffusions in polyhedral domains. Stochastic Process. Appl. 117 1014–1036.
  • [6] Budhiraja, A. and Lee, C. (2009). Stationary distribution convergence for generalized Jackson networks in heavy traffic. Math. Oper. Res. 34 45–56.
  • [7] Dai, J., Dieker, A. and Gao, X. (2012). Validity of heavy-traffic steady-state approximations in many-server queues with abandonment. Unpublished manuscript.
  • [8] Dai, J. G., He, S. and Tezcan, T. (2010). Many-server diffusion limits for $G/Ph/n+GI$ queues. Ann. Appl. Probab. 20 1854–1890.
  • [9] Dieker, A. B. and Gao, X. (2013). Positive recurrence of piecewise Ornstein–Uhlenbeck processes and common quadratic Lyapunov functions. Ann. Appl. Probab. 23 1291–1317.
  • [10] Ethier, S. N. and Kurtz, T. G. (1986). Markov Processes: Characterization and Convergence. Wiley, New York.
  • [11] Galtchouk, L. and Pergamenchtchikov, S. (2012). Geometric ergodicity for families of homogeneous Markov chains. Available at arXiv:1002.2341.
  • [12] Gamarnik, D. and Zeevi, A. (2006). Validity of heavy traffic steady-state approximation in generalized Jackson networks. Ann. Appl. Probab. 16 56–90.
  • [13] Gilbarg, D. and Trudinger, N. S. (2001). Elliptic Partial Differential Equations of Second Order. Springer, Berlin.
  • [14] Glynn, P. W. and Zeevi, A. (2008). Bounding stationary expectations of Markov processes. In Markov Processes and Related Topics: A Festschrift for Thomas G. Kurtz. Inst. Math. Stat. Collect. 4 195–214. IMS, Beachwood, OH.
  • [15] Gurvich, I., Huang, J. and Mandelbaum, A. (2014). Excursion-based universal approximations for the Erlang-A queue in steady-state. Math. Oper. Res. 39 325–373.
  • [16] Horn, R. A. and Johnson, C. R. (1994). Matrix Analysis. Cambridge Univ. Press, Cambridge.
  • [17] Karatzas, I. and Shreve, S. E. (1991). Brownian Motion and Stochastic Calculus, 2nd ed. Graduate Texts in Mathematics 113. Springer, New York.
  • [18] Khasminskii, R. (2012). Stochastic Stability of Differential Equations, 2nd ed. Stochastic Modelling and Applied Probability 66. Springer, Heidelberg.
  • [19] Klebaner, F. C. (2005). Introduction to Stochastic Calculus with Applications, 2nd ed. Imperial College Press, London.
  • [20] Meyn, S. P. and Tweedie, R. L. (1993). Stability of Markovian processes. III. Foster–Lyapunov criteria for continuous-time processes. Adv. in Appl. Probab. 25 518–548.
  • [21] Pardoux, E. and Veretennikov, A. Y. (2001). On the Poisson equation and diffusion approximation. I. Ann. Probab. 29 1061–1085.
  • [22] Qian, Z. and Zheng, W. (2004). A representation formula for transition probability densities of diffusions and applications. Stochastic Process. Appl. 111 57–76.
  • [23] Robert, P. (2003). Stochastic Networks and Queues, French ed. Applications of Mathematics (New York) 52. Springer, Berlin.