Journal of Applied Probability

Computing stationary expectations in level-dependent QBD processes

Hendrik Baumann and Werner Sandmann

Full-text: Access denied (no subscription detected)

We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text


Stationary expectations corresponding to long-run averages of additive functionals on level-dependent quasi-birth-and-death processes are considered. Special cases include long-run average costs or rewards, moments and cumulants of steady-state queueing network performance measures, and many others. We provide a matrix-analytic scheme for numerically computing such stationary expectations without explicitly computing the stationary distribution of the process, which yields an algorithm that is as quick as its counterparts for stationary distributions but requires far less computer storage. Specific problems arising in the case of infinite state spaces are discussed and the application of the algorithm is demonstrated by a queueing network example.

Article information

J. Appl. Probab., Volume 50, Number 1 (2013), 151-165.

First available in Project Euclid: 20 March 2013

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 60J22: Computational methods in Markov chains [See also 65C40]
Secondary: 60J27: Continuous-time Markov processes on discrete state spaces 60J28: Applications of continuous-time Markov processes on discrete state spaces

Level-dependent quasi-birth-and-death process matrix-analytic computation long-run average additive functional stationary expectation memory-efficient algorithm


Baumann, Hendrik; Sandmann, Werner. Computing stationary expectations in level-dependent QBD processes. J. Appl. Probab. 50 (2013), no. 1, 151--165. doi:10.1239/jap/1363784430.

Export citation


  • Asmussen, S. (2003). Applied Probability and Queues, 2nd edn. Springer, New York.
  • Baumann, H. and Sandmann, W. (2010). Numerical solution of level dependent quasi-birth-and-death processes. Procedia Comput. Sci. 1, 1561–1569.
  • Baumann, H. and Sandmann, W. (2012). Steady state analysis of level dependent quasi-birth-and-death processes with catastrophes. Comput. Operat. Res. 39, 413–423.
  • Bright, L. and Taylor, P. G. (1995). Calculating the equilibrium distribution in level dependent quasi-birth-and-death processes. Commun. Statist. Stoch. Models 11, 497–525.
  • Dayar, T., Sandmann, W., Spieler, D. and Wolf, V. (2011). Infinite level-dependent QBD processes and matrix-analytic solutions for stochastic chemical kinetics. Adv. Appl. Prob. 43, 1005–1026.
  • Gaver, D. P., Jacobs, P. A. and Latouche, G. (1984). Finite birth-and-death models in randomly changing environments. Adv. Appl. Prob. 16, 715–731.
  • Glynn, P. W. and Zeevi, A. (2008). Bounding stationary expectations of Markov processes. In Markov Processes and Related Topics: A Festschrift for Thomas G. Kurtz (Inst. Math. Statist. Collect. 4), Institute of Mathematical Statistics, Beachwood, OH, pp. 195–214.
  • Golub, G. H. and Van Loan, C. F. (1996). Matrix Computations, 3rd edn. Johns Hopkins University Press, Baltimore, MD.
  • Hanschke, T. (1999). A matrix continued fraction algorithm for the multiserver repeated order queue. Math. Comput. Modelling 30, 159–170.
  • Knuth, D. E. (1998). The Art of Computer Programming, Vol. 2, 3rd edn. Addison-Wesley.
  • Kumar, S. and Kumar, P. R. (1994). Performance bounds for queueing networks and scheduling policies. IEEE Trans. Automatic Control 39, 1600–1611.
  • Latouche, G. and Ramaswami, V. (1999). Introduction to Matrix Analytic Methods in Stochastic Modeling. SIAM, Philadelphia, PA.
  • Morrison, J. R. and Kumar, P. R. (2008). Computational performance bounds for Markov chains with applications. IEEE Trans. Automatic Control 53, 1306–1311.
  • Neuts, M. F. (1981). Matrix-Geometric Solutions in Stochastic Models. Johns Hopkins University Press, Baltimore, MD.
  • Serfozo, R. (2009). Basics of Applied Stochastic Processes. Springer, Berlin.
  • Stewart, W. J. (1994). Introduction to the Numerical Solution of Markov Chains. Princeton University Press.
  • Thorne, J. (1997). An investigation of algorithms for calculating the fundamental matrices in level dependent quasi birth death processes. Honours thesis, The University of Adelaide.