## Bernoulli

• Bernoulli
• Volume 5, Number 3 (1999), 535-569.

### Asymptotic behaviour of stationary distributions for countable Markov chains, with some applications

#### Abstract

Let ${ Z n ,n≥0}$ be an aperiodic irreducible recurrent (not necessarily positive recurrent) Markov chain taking values on a countable unbounded subset $S$ of $R d$, $π (⋅)$ its invariant measure and $f$ is a non-negative function defined on $S$. We first find sufficient conditions under which $∫ S f(z)π(dz)=∞$ (the corresponding result for the finiteness of $∫ S f(z)π(dz)$ was obtained by Tweedie). Then we obtain lower and upper bounds for the values of the invariant measure $π$ on the subsets $B$ of $S$, that is, $π (B)$. These bounds are expressed in terms of first passage probabilities and the first exit time from $B$. We also show how to estimate the latter quantities using sub- or supermartingale techniques. The results are finally illustrated for driftless reflected random walks in $ℤ + 2$ and for Markov chains on non-negative reals with asymptotically small drift of Lamperti type. In both cases we obtain very precise information on the asymptotic behaviour of their stationary measures.

#### Article information

Source
Bernoulli, Volume 5, Number 3 (1999), 535-569.

Dates
First available in Project Euclid: 27 February 2007

https://projecteuclid.org/euclid.bj/1172617202

Mathematical Reviews number (MathSciNet)
MR1693596

Zentralblatt MATH identifier
0948.60068

#### Citation

Aspandiiarov, Sanjar; Iasnogorodski, Roudolf. Asymptotic behaviour of stationary distributions for countable Markov chains, with some applications. Bernoulli 5 (1999), no. 3, 535--569. https://projecteuclid.org/euclid.bj/1172617202

#### References

• [1] Asmussen, S. (1987) Applied Probability and Queues. Chichester: Wiley.
• [2] Aspandiiarov, S (1994) Some properties of Brownian motion and reflected Markov chains. Doctoral thesis. Université Paris VI, Paris.
• [3] Aspandiiarov, S. and Iasnogorodski, R. (1995) General results on the stationary distributions for countable Markov chains and their applications. Prepublication no. 273 of the Laboratoire de Probabilités, Université Paris VI.
• [4] Aspandiiarov, S. and Iasnogorodski, R. (1997) Tails of passage-time for non-negative stochastic processes and an application to stochastic processes with boundary reflection in a wedge. Stochastic Process. Appl., 66, 115-145.
• [5] Aspandiiarov, S. and Iasnogorodski, R. (1998) General criteria of integrability of functions of passagetimes for non-negative stochastic processes and their applications. Theory Probab. Appl., 43, 509-539.
• [6] Aspandiiarov, S., Iasnogorodski, R. and Menshikov, M, (1996) Passage-time moments for non-negative stochastic processes and an application to reflected random walks in a quadrant. Ann. Probab., 24, 932-960.
• [7] Borovkov, A.A., Fayolle, G. and Korshunov, D.A. (1992) Transient phenomena for Markov chains and their applications. J. Appl. Probab., 24, 322-342.
• [8] Chung, K.L. (1967) Markov Chains with Stationary Transition Probabilities. Berlin: Springer-Verlag.
• [9] Fayolle, G., Malyshev, V. and Menshikov, M. (1992) Random walks in a quarter plane with zero drifts. Ann. Inst. H. Poincaré Probab. Statist., 28, 179-195.
• [10] Fayolle, G., Malyshev, V. and Menshikov, M. (1994) Topics in the Constructive Theory of Countable Markov Chains. Cambridge: Cambridge University Press.
• [11] Foster, F.G. (1953) On stochastic matrices associated with certain queueing processes. Ann. Math. Statist., 24, 355-360.
• [12] Lamperti, J. (1963) Criteria for stochastic processes II: Passage-time moments. J. Math. Anal. Appl., 7, 127-145.
• [13] Menshikov, M. and Popov, S. (1995) Exact power estimates for countable Markov chains. Markov Process. Related Fields, 1, 57-78.
• [14] Meyn, S.P. and Tweedie, R.L. (1993) Markov Chains and Stochastic Stability. Berlin: Springer-Verlag.
• [15] Nummelin, E. and Tuominnen, P. (1982) Geometric ergodicity of Harris recurrent Markov chains with applications to renewal theory. Stochastic Process. Appl., 12, 187-202.
• [16] Nummelin, E. and Tweedie, R.L. (1978) Geometric ergodicity and R-positivity for general Markov chains. Ann. Probab., 6, 404-420.
• [17] Nummelin, E. and Tweedie, R.L. (1994) Subgeometric rates of convergence of f-ergodic Markov chains. Adv. Appl. Probab., 26, 775-798.
• [18] Tweedie, R.L. (1983) The existence of moments for stationary Markov chains. J. Appl. Probab., 28, 191-196.
• [19] Williams, R.J. (1985) Recurrence classification for reflected Brownian motion in a wedge, Ann. Probab., 13, 758-778.