The Annals of Applied Probability

Diffusion models and steady-state approximations for exponentially ergodic Markovian queues

Itai Gurvich

Full-text: Open access

Enhanced PDF (313 KB)

Abstract

Motivated by queues with many servers, we study Brownian steady-state approximations for continuous time Markov chains (CTMCs). Our approximations are based on diffusion models (rather than a diffusion limit) whose steady-state, we prove, approximates that of the Markov chain with notable precision. Strong approximations provide such “limitless” approximations for process dynamics. Our focus here is on steady-state distributions, and the diffusion model that we propose is tractable relative to strong approximations.

Within an asymptotic framework, in which a scale parameter $n$ is taken large, a uniform (in the scale parameter) Lyapunov condition imposed on the sequence of diffusion models guarantees that the gap between the steady-state moments of the diffusion and those of the properly centered and scaled CTMCs shrinks at a rate of $\sqrt{n}$.

Our proofs build on gradient estimates for solutions of the Poisson equations associated with the (sequence of) diffusion models and on elementary martingale arguments. As a by-product of our analysis, we explore connections between Lyapunov functions for the fluid model, the diffusion model and the CTMC.

Article information

Source
Ann. Appl. Probab., Volume 24, Number 6 (2014), 2527-2559.

Dates
First available in Project Euclid: 26 August 2014

Permanent link to this document
https://projecteuclid.org/euclid.aoap/1409058039

Digital Object Identifier
doi:10.1214/13-AAP984

Mathematical Reviews number (MathSciNet)
MR3262510

Zentralblatt MATH identifier
1333.60194

Subjects
Primary: 60K25: Queueing theory [See also 68M20, 90B22] 90B20: Traffic problems 90B36: Scheduling theory, stochastic [See also 68M20] 49L20: Dynamic programming method 60F17: Functional limit theorems; invariance principles

Keywords
Markovian queues steady-state many servers heavy-traffic Halfin–Whitt regime steady state approximations strong approximations for queues

Citation

Gurvich, Itai. Diffusion models and steady-state approximations for exponentially ergodic Markovian queues. Ann. Appl. Probab. 24 (2014), no. 6, 2527--2559. doi:10.1214/13-AAP984. https://projecteuclid.org/euclid.aoap/1409058039


Export citation

References

  • [1] Ata, B. and Gurvich, I. (2012). On optimality gaps in the Halfin–Whitt regime. Ann. Appl. Probab. 22 407–455.
    Mathematical Reviews (MathSciNet): MR2932551
    Digital Object Identifier: doi:10.1214/11-AAP777
    Project Euclid: euclid.aoap/1328623704
  • [2] Baxendale, P. H. (2005). Renewal theory and computable convergence rates for geometrically ergodic Markov chains. Ann. Appl. Probab. 15 700–738.
    Mathematical Reviews (MathSciNet): MR2114987
    Digital Object Identifier: doi:10.1214/105051604000000710
    Project Euclid: euclid.aoap/1107271665
  • [3] Borkar, V. and Budhiraja, A. (2004/05). Ergodic control for constrained diffusions: Characterization using HJB equations. SIAM J. Control Optim. 43 1467–1492.
    Mathematical Reviews (MathSciNet): MR2124281
    Digital Object Identifier: doi:10.1137/S0363012902417619
  • [4] Brémaud, P. (1981). Point Processes and Queues: Martingale Dynamics. Springer, New York.
    Mathematical Reviews (MathSciNet): MR636252
  • [5] Budhiraja, A. and Lee, C. (2007). Long time asymptotics for constrained diffusions in polyhedral domains. Stochastic Process. Appl. 117 1014–1036.
    Mathematical Reviews (MathSciNet): MR2340877
    Digital Object Identifier: doi:10.1016/j.spa.2006.11.007
  • [6] Budhiraja, A. and Lee, C. (2009). Stationary distribution convergence for generalized Jackson networks in heavy traffic. Math. Oper. Res. 34 45–56.
    Mathematical Reviews (MathSciNet): MR2542988
    Digital Object Identifier: doi:10.1287/moor.1080.0353
  • [7] Dai, J., Dieker, A. and Gao, X. (2012). Validity of heavy-traffic steady-state approximations in many-server queues with abandonment. Unpublished manuscript.
  • [8] Dai, J. G., He, S. and Tezcan, T. (2010). Many-server diffusion limits for $G/Ph/n+GI$ queues. Ann. Appl. Probab. 20 1854–1890.
    Mathematical Reviews (MathSciNet): MR2724423
    Digital Object Identifier: doi:10.1214/09-AAP674
    Project Euclid: euclid.aoap/1282747403
  • [9] Dieker, A. B. and Gao, X. (2013). Positive recurrence of piecewise Ornstein–Uhlenbeck processes and common quadratic Lyapunov functions. Ann. Appl. Probab. 23 1291–1317.
    Mathematical Reviews (MathSciNet): MR3098433
    Digital Object Identifier: doi:10.1214/12-AAP870
    Project Euclid: euclid.aoap/1371834029
  • [10] Ethier, S. N. and Kurtz, T. G. (1986). Markov Processes: Characterization and Convergence. Wiley, New York.
    Mathematical Reviews (MathSciNet): MR838085
  • [11] Galtchouk, L. and Pergamenchtchikov, S. (2012). Geometric ergodicity for families of homogeneous Markov chains. Available at arXiv:1002.2341.
    arXiv: 1002.2341
  • [12] Gamarnik, D. and Zeevi, A. (2006). Validity of heavy traffic steady-state approximation in generalized Jackson networks. Ann. Appl. Probab. 16 56–90.
    Mathematical Reviews (MathSciNet): MR2209336
    Digital Object Identifier: doi:10.1214/105051605000000638
    Project Euclid: euclid.aoap/1141654281
  • [13] Gilbarg, D. and Trudinger, N. S. (2001). Elliptic Partial Differential Equations of Second Order. Springer, Berlin.
    Mathematical Reviews (MathSciNet): MR1814364
  • [14] Glynn, P. W. and Zeevi, A. (2008). Bounding stationary expectations of Markov processes. In Markov Processes and Related Topics: A Festschrift for Thomas G. Kurtz. Inst. Math. Stat. Collect. 4 195–214. IMS, Beachwood, OH.
    Mathematical Reviews (MathSciNet): MR2574232
    Digital Object Identifier: doi:10.1214/074921708000000381
  • [15] Gurvich, I., Huang, J. and Mandelbaum, A. (2014). Excursion-based universal approximations for the Erlang-A queue in steady-state. Math. Oper. Res. 39 325–373.
    Mathematical Reviews (MathSciNet): MR3205550
    Digital Object Identifier: doi:10.1287/moor.2013.0606
  • [16] Horn, R. A. and Johnson, C. R. (1994). Matrix Analysis. Cambridge Univ. Press, Cambridge.
  • [17] Karatzas, I. and Shreve, S. E. (1991). Brownian Motion and Stochastic Calculus, 2nd ed. Graduate Texts in Mathematics 113. Springer, New York.
    Mathematical Reviews (MathSciNet): MR1121940
  • [18] Khasminskii, R. (2012). Stochastic Stability of Differential Equations, 2nd ed. Stochastic Modelling and Applied Probability 66. Springer, Heidelberg.
    Mathematical Reviews (MathSciNet): MR2894052
  • [19] Klebaner, F. C. (2005). Introduction to Stochastic Calculus with Applications, 2nd ed. Imperial College Press, London.
    Mathematical Reviews (MathSciNet): MR2160228
  • [20] Meyn, S. P. and Tweedie, R. L. (1993). Stability of Markovian processes. III. Foster–Lyapunov criteria for continuous-time processes. Adv. in Appl. Probab. 25 518–548.
    Mathematical Reviews (MathSciNet): MR1234295
    Digital Object Identifier: doi:10.2307/1427522
  • [21] Pardoux, E. and Veretennikov, A. Y. (2001). On the Poisson equation and diffusion approximation. I. Ann. Probab. 29 1061–1085.
    Mathematical Reviews (MathSciNet): MR1872736
    Digital Object Identifier: doi:10.1214/aop/1015345596
    Project Euclid: euclid.aop/1015345596
  • [22] Qian, Z. and Zheng, W. (2004). A representation formula for transition probability densities of diffusions and applications. Stochastic Process. Appl. 111 57–76.
    Mathematical Reviews (MathSciNet): MR2049569
    Digital Object Identifier: doi:10.1016/j.spa.2003.12.004
  • [23] Robert, P. (2003). Stochastic Networks and Queues, French ed. Applications of Mathematics (New York) 52. Springer, Berlin.
    Mathematical Reviews (MathSciNet): MR1996883

The Institute of Mathematical Statistics

The Annals of Applied Probability

New content alerts

Email RSS ToC RSS Article
  • You have access to this content.
  • You have partial access to this content.
  • You do not have access to this content.
Turn Off MathJax

What is MathJax?

More like this

+ See more