Characterization and sufficient conditions for normed ergodicity of Markov chains
A. A. Borovkov and A. Hordijk
Source: Adv. in Appl. Probab. Volume 36, Number 1 (2004), 227-242.
Abstract
Normed ergodicity is a type of strong ergodicity for which convergence of the nth step transition operator to the stationary operator holds in the operator norm. We derive a new characterization of normed ergodicity and we clarify its relation with exponential ergodicity. The existence of a Lyapunov function together with two conditions on the uniform integrability of the increments of the Markov chain is shown to be a sufficient condition for normed ergodicity. Conversely, the sufficient conditions are also almost necessary.
Primary Subjects: 60J05
Secondary Subjects: 90B22
Keywords: Markov chain; normed ergodicity; Lyapunov function
Full-text: Access denied (no subscription detected)
We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber.
If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text
Links and Identifiers
Permanent link to this document: http://projecteuclid.org/euclid.aap/1077134471
Mathematical Reviews number (MathSciNet):
MR2035781
Zentralblatt MATH identifier:
02067394
Digital Object Identifier: doi:10.1239/aap/1077134471
References
Borovkov, A. A. (1998). Ergodicity and Stability of Stochastic Processes. John Wiley, Chichester.
Mathematical Reviews (MathSciNet):
MR1658404
Zentralblatt MATH:
0917.60005
Borovkov, A. A. and Hordijk, A. (2000). On normed ergodicity of Markov chains. Tech. Rep. MI 2000--40, Leiden University.
Borovkov, A. A. and Hordijk, A. (2004). On linear Lyapunov functions and normed ergodicity for stochastic networks. In preparation.
Dekker, R. and Hordijk, A. (1988). Average, sensitive and Blackwell optimal policies in denumerable Markov decision chains with unbounded rewards. Math. Operat. Res. 13, 395--421.
Mathematical Reviews (MathSciNet):
MR961801
Dekker, R., Hordijk, A. and Spieksma, F. M. (1994). On the relation between recurrence and ergodicity properties in denumerable Markov decision chains. Math. Operat. Res. 19, 539--559.
Mathematical Reviews (MathSciNet):
MR1288886
Fayolle, G., Malyshev, V. A. and Menshikov, M. V. (1995). Topics in the Constructive Theory of Countable Markov Chains. Cambridge University Press.
Mathematical Reviews (MathSciNet):
MR1331145
Zentralblatt MATH:
0823.60053
Federgruen, A., Hordijk, A. and Tijms, H. C. (1978). Recurrence conditions in denumerable state Markov decision processes. In Dynamic Programming and Its Applications, ed. M. L. Puterman, Academic Press, New York, pp. 3--22.
Mathematical Reviews (MathSciNet):
MR537873
Zentralblatt MATH:
0458.90079
Federgruen, A., Hordijk, A. and Tijms, H. C. (1978). A note on simultaneous recurrence conditions on a set of denumerable stochastic matrices. J. Appl. Prob. 15, 842--847.
Mathematical Reviews (MathSciNet):
MR511062
Heidergott, B. and Hordijk, A. (2003). Taylor series expansions for stationary Markov chains. Adv. Appl. Prob. 35, 1046--1070.
Mathematical Reviews (MathSciNet):
MR2014269
Digital Object Identifier: doi:10.1239/aap/1067436334
Heidergott, B., Hordijk, A. and Weisshaupt, H. (2002). Measure-valued differentiation for stationary Markov chains. EURANDOM report 2002--027.
Hordijk, A. (1974). Dynamic Programming and Markov Potential Theory (Math. Centre Tract 51). CWI, Amsterdam.
Mathematical Reviews (MathSciNet):
MR432227
Zentralblatt MATH:
0284.49012
Hordijk, A. and Dekker, R. (1983). Average, sensitive and Blackwell optimal policies in denumerable Markov decision chains with unbounded rewards. Report No. 83--36, Institute of Applied Mathematics and Computing Science, Leiden University.
Mathematical Reviews (MathSciNet):
MR838768
Zentralblatt MATH:
0517.90088
Hordijk, A. and Spieksma, F. M. (1992). On ergodicity and recurrence properties of a Markov chain with an application to an open Jackson network. Adv. Appl. Prob. 24, 343--376.
Mathematical Reviews (MathSciNet):
MR1167263
Hordijk, A. and Yushkevich, A. A. (1999). Blackwell optimality in the class of all policies in Markov decision chains with a Borel state space and unbounded rewards. Math. Meth. Operat. Res. 50, 421--448.
Mathematical Reviews (MathSciNet):
MR1731301
Digital Object Identifier: doi:10.1007/s001860050079
Hordijk, A., Spieksma, F. M. and Tweedie, R. L. (1995). Uniform stability for general state space Markov decision processes. Tech. Rep., Leiden University.
Kartoschov, N. (1985). Inequalities in theorems of ergodicity and stability for Markov chains with common phase space. Theory Prob. Appl. 30, 247--259.
Meyn, S. P. and Tweedie, R. L. (1993). Markov chains and stochastic stability. Springer, London.
Mathematical Reviews (MathSciNet):
MR1287609
Zentralblatt MATH:
0925.60001
Neveu, J. (1965). Mathematical Foundations of the Calculus of Probability. Holden Day, San Francisco.
Mathematical Reviews (MathSciNet):
MR198505
Zentralblatt MATH:
0137.11301
Spieksma, F. M. and Tweedie, R. L. (1994). Strenghtening ergodicity to geometric ergodicity for Markov chains. Stoch. Models 10, 45--75.
Mathematical Reviews (MathSciNet):
MR1259854
Advances in Applied Probability
