Advances in Applied Probability

Taylor series expansions for stationary Markov chains

Bernd Heidergott and Arie Hordijk

Full-text: Access denied (no subscription detected) We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text

Abstract

We study Taylor series expansions of stationary characteristics of general-state-space Markov chains. The elements of the Taylor series are explicitly calculated and a lower bound for the radius of convergence of the Taylor series is established. The analysis provided in this paper applies to the case where the stationary characteristic is given through an unbounded sample performance function such as the second moment of the stationary waiting time in a queueing system.

Article information

Source
Adv. in Appl. Probab. Volume 35, Number 4 (2003), 1046-1070.

Dates
First available in Project Euclid: 29 October 2003

Permanent link to this document
http://projecteuclid.org/euclid.aap/1067436334

Digital Object Identifier
doi:10.1239/aap/1067436334

Mathematical Reviews number (MathSciNet)
MR2014269

Zentralblatt MATH identifier
02052112

Subjects
Primary: 60J10: Markov chains (discrete-time Markov processes on discrete state spaces)
Secondary: 90C31: Sensitivity, stability, parametric optimization

Keywords
Markov chain Taylor series measure-valued differentiation deviation matrix

Citation

Heidergott, Bernd; Hordijk, Arie. Taylor series expansions for stationary Markov chains. Adv. in Appl. Probab. 35 (2003), no. 4, 1046--1070. doi:10.1239/aap/1067436334. http://projecteuclid.org/euclid.aap/1067436334.


Export citation

References

  • Ayhan, H. and Baccelli, F. (2001). Expansions for joint Laplace transforms of stationary waiting times in (max,$+$)-linear systems with Poisson input. Queueing Systems 37, 291--328.
  • Ayhan, H. and Seo, D. (2001). Laplace transform and moments of waiting times in Poisson driven (max,$+$)-linear systems. Queueing Systems 37, 405--436.
  • Ayhan, H. and Seo, D. (2002). Tail probability of transient and stationary waiting times in (max,$+$)-linear systems. IEEE Trans. Automatic Control 47, 151--157.
  • Baccelli, F. and Schmidt, V. (1996). Taylor series expansions for Poisson-driven (max,$+$)-linear systems. Ann. Appl. Prob. 6, 138--185.
  • Baccelli, F., Hasenfuss, S. and Schmidt, V. (1997). Transient and stationary waiting times in (max,$+$)-linear systems with Poisson input. Queueing Systems 26, 301--342.
  • Baccelli, F., Hasenfuss, S. and Schmidt, V. (1998). Expansions for steady-state characteristics of (max,$+$)-linear systems. Commun. Statist. Stoch. Models 14, 1--24.
  • Borovkov, A. and Hordijk, A. (2004). Characterization and sufficient conditions for normed ergodicity of Markov chains. To appear in Adv. Appl. Prob. 36, No. 1.
  • Cao, X.-R. (1998). The Maclaurin series for performance functions of Markov chains. Adv. Appl. Prob. 30, 676--692.
  • Cassandras, C. and Lafortune, S. (1999). Introduction to Discrete Event Systems. Kluwer, Norwell, MA.
  • Coolen-Schrijner, P. and van Doorn, E. A. (2002). The deviation matrix of a continuous-time Markov chain. Prob. Eng. Inf. Sci. 16, 351--366.
  • Dekker, R. and Hordijk, A. (1988). Average, sensitive and Blackwell optimal policies in denumerable Markov decision chains with unbounded rewards. Math. Operat. Res. 13, 395--421.
  • Dekker, R., Hordijk, A. and Spieksma, F. M. (1994). On the relation between recurrence and ergodicity properties in denumerable Markov decision chains. Math. Operat. Res. 19, 539--559.
  • Gong, W.-B. and Hu, J.-Q. (1992). The Maclaurin series of the GI/G/1 queue. J. Appl. Prob. 29, 176--184.
  • Heidergott, B. and Vázquez-Abad, F. (2000). Measure-valued differentiation for stochastic processes: the finite horizon case. EURANDOM Report 2000-033. Available at http://staff.feweb.vu.nl/bheidergott/.
  • Heidergott, B., Hordijk, A. and Weisshaupt, H. (2002). Derivatives of Markov kernels and their Jordan decomposition. EURANDOM Report 2003-001. Available at http://staff.feweb.vu.nl/bheidergott/.
  • Heidergott, B., Hordijk, A. and Weisshaupt, H. (2002). Measure-valued differentiation for stationary Markov chains. EURANDOM Report 2002-027. Available at http://staff.feweb.vu.nl/bheidergott/.
  • Ho, Y. and Cao, X. (1991). Perturbation Analysis of Discrete Event Systems. Kluwer, Boston, MA.
  • Hordijk, A. and Dekker, R. (1983). Average, sensitive and Blackwell optimal policies in denumerable Markov decision chains with unbounded rewards. Rep. 83--36, Institute of Applied Mathematics and Computing Science, Leiden University.
  • Hordijk, A. and Puterman, M. L. (1987). On the convergence of policy iteration in finite state undiscounted Markov decision processes: the unichain case. Math. Operat. Res. 12, 163--176.
  • Hordijk, A. and Spieksma, F. M. (1992). On ergodicity and recurrence properties of a Markov chain with an application to an open Jackson network. Adv. Appl. Prob. 24, 343--376.
  • Hu, J.-Q. (1995). Analyticity of single-server queues in light traffic. Queueing Systems 19, 63--80.
  • Hu, J.-Q. (1996). The departure process of the GI/G/1 queue and its Maclaurin series. Operat. Res. 44, 810--815.
  • Kartoschov, N. (1985). Inequalities in theorems of ergodicity and stability for Markov chains with common phase space. Theory Prob. Appl. 30, 247--259.
  • Koole, G. M. (1998). The deviation matrix of the M/M/1/$\infty $ and M/M/1/$N$ queue, with applications to controlled queueing models. In Proc. 37th IEEE Conf. Decision Control (Tampa, FL), IEEE Press, pp. 56--59.
  • Koole, G. M. and Spieksma, F. M. (2001). On deviation matrices for birth--death process. Prob. Eng. Inf. Sci. 15, 239--258.
  • Meyn, S. P. and Tweedie, R. L. (1993). Markov Chains and Stochastic Stability. Springer, London.
  • Pflug, G. (1996). Optimisation of Stochastic Models. Kluwer, Boston, MA.
  • Van den Hout, W. (1996). The power-series-algorithm. Doctoral Thesis, Center for Economic Research, Tilburg University.
  • Zhu, Y. and Li, H. (1993). The Maclaurin expansion for a G/GI/1 queue with Markov-modulated arrivals and services. Queueing Systems 14, 125--134.
  • Zazanis, M. (1992). Analyticity of Poisson-driven stochastic systems. Adv. Appl. Prob. 24, 532--541.

See also