The Annals of Applied Probability

Diffusion models and steady-state approximations for exponentially ergodic Markovian queues

Itai Gurvich

Full-text: Access denied (no subscription detected)

We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text


Motivated by queues with many servers, we study Brownian steady-state approximations for continuous time Markov chains (CTMCs). Our approximations are based on diffusion models (rather than a diffusion limit) whose steady-state, we prove, approximates that of the Markov chain with notable precision. Strong approximations provide such “limitless” approximations for process dynamics. Our focus here is on steady-state distributions, and the diffusion model that we propose is tractable relative to strong approximations.

Within an asymptotic framework, in which a scale parameter $n$ is taken large, a uniform (in the scale parameter) Lyapunov condition imposed on the sequence of diffusion models guarantees that the gap between the steady-state moments of the diffusion and those of the properly centered and scaled CTMCs shrinks at a rate of $\sqrt{n}$.

Our proofs build on gradient estimates for solutions of the Poisson equations associated with the (sequence of) diffusion models and on elementary martingale arguments. As a by-product of our analysis, we explore connections between Lyapunov functions for the fluid model, the diffusion model and the CTMC.

Article information

Ann. Appl. Probab. Volume 24, Number 6 (2014), 2527-2559.

First available in Project Euclid: 26 August 2014

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 60K25: Queueing theory [See also 68M20, 90B22] 90B20: Traffic problems 90B36: Scheduling theory, stochastic [See also 68M20] 49L20: Dynamic programming method 60F17: Functional limit theorems; invariance principles

Markovian queues steady-state many servers heavy-traffic Halfin–Whitt regime steady state approximations strong approximations for queues


Gurvich, Itai. Diffusion models and steady-state approximations for exponentially ergodic Markovian queues. Ann. Appl. Probab. 24 (2014), no. 6, 2527--2559. doi:10.1214/13-AAP984.

Export citation


  • [1] Ata, B. and Gurvich, I. (2012). On optimality gaps in the Halfin–Whitt regime. Ann. Appl. Probab. 22 407–455.
  • [2] Baxendale, P. H. (2005). Renewal theory and computable convergence rates for geometrically ergodic Markov chains. Ann. Appl. Probab. 15 700–738.
  • [3] Borkar, V. and Budhiraja, A. (2004/05). Ergodic control for constrained diffusions: Characterization using HJB equations. SIAM J. Control Optim. 43 1467–1492.
  • [4] Brémaud, P. (1981). Point Processes and Queues: Martingale Dynamics. Springer, New York.
  • [5] Budhiraja, A. and Lee, C. (2007). Long time asymptotics for constrained diffusions in polyhedral domains. Stochastic Process. Appl. 117 1014–1036.
  • [6] Budhiraja, A. and Lee, C. (2009). Stationary distribution convergence for generalized Jackson networks in heavy traffic. Math. Oper. Res. 34 45–56.
  • [7] Dai, J., Dieker, A. and Gao, X. (2012). Validity of heavy-traffic steady-state approximations in many-server queues with abandonment. Unpublished manuscript.
  • [8] Dai, J. G., He, S. and Tezcan, T. (2010). Many-server diffusion limits for $G/Ph/n+GI$ queues. Ann. Appl. Probab. 20 1854–1890.
  • [9] Dieker, A. B. and Gao, X. (2013). Positive recurrence of piecewise Ornstein–Uhlenbeck processes and common quadratic Lyapunov functions. Ann. Appl. Probab. 23 1291–1317.
  • [10] Ethier, S. N. and Kurtz, T. G. (1986). Markov Processes: Characterization and Convergence. Wiley, New York.
  • [11] Galtchouk, L. and Pergamenchtchikov, S. (2012). Geometric ergodicity for families of homogeneous Markov chains. Available at arXiv:1002.2341.
  • [12] Gamarnik, D. and Zeevi, A. (2006). Validity of heavy traffic steady-state approximation in generalized Jackson networks. Ann. Appl. Probab. 16 56–90.
  • [13] Gilbarg, D. and Trudinger, N. S. (2001). Elliptic Partial Differential Equations of Second Order. Springer, Berlin.
  • [14] Glynn, P. W. and Zeevi, A. (2008). Bounding stationary expectations of Markov processes. In Markov Processes and Related Topics: A Festschrift for Thomas G. Kurtz. Inst. Math. Stat. Collect. 4 195–214. IMS, Beachwood, OH.
  • [15] Gurvich, I., Huang, J. and Mandelbaum, A. (2014). Excursion-based universal approximations for the Erlang-A queue in steady-state. Math. Oper. Res. 39 325–373.
  • [16] Horn, R. A. and Johnson, C. R. (1994). Matrix Analysis. Cambridge Univ. Press, Cambridge.
  • [17] Karatzas, I. and Shreve, S. E. (1991). Brownian Motion and Stochastic Calculus, 2nd ed. Graduate Texts in Mathematics 113. Springer, New York.
  • [18] Khasminskii, R. (2012). Stochastic Stability of Differential Equations, 2nd ed. Stochastic Modelling and Applied Probability 66. Springer, Heidelberg.
  • [19] Klebaner, F. C. (2005). Introduction to Stochastic Calculus with Applications, 2nd ed. Imperial College Press, London.
  • [20] Meyn, S. P. and Tweedie, R. L. (1993). Stability of Markovian processes. III. Foster–Lyapunov criteria for continuous-time processes. Adv. in Appl. Probab. 25 518–548.
  • [21] Pardoux, E. and Veretennikov, A. Y. (2001). On the Poisson equation and diffusion approximation. I. Ann. Probab. 29 1061–1085.
  • [22] Qian, Z. and Zheng, W. (2004). A representation formula for transition probability densities of diffusions and applications. Stochastic Process. Appl. 111 57–76.
  • [23] Robert, P. (2003). Stochastic Networks and Queues, French ed. Applications of Mathematics (New York) 52. Springer, Berlin.