Bernoulli

  • Bernoulli
  • Volume 5, Number 3 (1999), 535-569.

Asymptotic behaviour of stationary distributions for countable Markov chains, with some applications

Sanjar Aspandiiarov and Roudolf Iasnogorodski

Full-text: Open access

Abstract

Let { Z n ,n0} be an aperiodic irreducible recurrent (not necessarily positive recurrent) Markov chain taking values on a countable unbounded subset S of R d , π () its invariant measure and f is a non-negative function defined on S . We first find sufficient conditions under which S f(z)π(dz)= (the corresponding result for the finiteness of S f(z)π(dz) was obtained by Tweedie). Then we obtain lower and upper bounds for the values of the invariant measure π on the subsets B of S , that is, π (B) . These bounds are expressed in terms of first passage probabilities and the first exit time from B . We also show how to estimate the latter quantities using sub- or supermartingale techniques. The results are finally illustrated for driftless reflected random walks in + 2 and for Markov chains on non-negative reals with asymptotically small drift of Lamperti type. In both cases we obtain very precise information on the asymptotic behaviour of their stationary measures.

Article information

Source
Bernoulli, Volume 5, Number 3 (1999), 535-569.

Dates
First available in Project Euclid: 27 February 2007

Permanent link to this document
https://projecteuclid.org/euclid.bj/1172617202

Mathematical Reviews number (MathSciNet)
MR1693596

Zentralblatt MATH identifier
0948.60068

Keywords
occupation time recurrent Markov chain reflected random walk stationary measure submartingale supermartingale

Citation

Aspandiiarov, Sanjar; Iasnogorodski, Roudolf. Asymptotic behaviour of stationary distributions for countable Markov chains, with some applications. Bernoulli 5 (1999), no. 3, 535--569. https://projecteuclid.org/euclid.bj/1172617202


Export citation

References

  • [1] Asmussen, S. (1987) Applied Probability and Queues. Chichester: Wiley.
  • [2] Aspandiiarov, S (1994) Some properties of Brownian motion and reflected Markov chains. Doctoral thesis. Université Paris VI, Paris.
  • [3] Aspandiiarov, S. and Iasnogorodski, R. (1995) General results on the stationary distributions for countable Markov chains and their applications. Prepublication no. 273 of the Laboratoire de Probabilités, Université Paris VI.
  • [4] Aspandiiarov, S. and Iasnogorodski, R. (1997) Tails of passage-time for non-negative stochastic processes and an application to stochastic processes with boundary reflection in a wedge. Stochastic Process. Appl., 66, 115-145.
  • [5] Aspandiiarov, S. and Iasnogorodski, R. (1998) General criteria of integrability of functions of passagetimes for non-negative stochastic processes and their applications. Theory Probab. Appl., 43, 509-539.
  • [6] Aspandiiarov, S., Iasnogorodski, R. and Menshikov, M, (1996) Passage-time moments for non-negative stochastic processes and an application to reflected random walks in a quadrant. Ann. Probab., 24, 932-960.
  • [7] Borovkov, A.A., Fayolle, G. and Korshunov, D.A. (1992) Transient phenomena for Markov chains and their applications. J. Appl. Probab., 24, 322-342.
  • [8] Chung, K.L. (1967) Markov Chains with Stationary Transition Probabilities. Berlin: Springer-Verlag.
  • [9] Fayolle, G., Malyshev, V. and Menshikov, M. (1992) Random walks in a quarter plane with zero drifts. Ann. Inst. H. Poincaré Probab. Statist., 28, 179-195.
  • [10] Fayolle, G., Malyshev, V. and Menshikov, M. (1994) Topics in the Constructive Theory of Countable Markov Chains. Cambridge: Cambridge University Press.
  • [11] Foster, F.G. (1953) On stochastic matrices associated with certain queueing processes. Ann. Math. Statist., 24, 355-360.
  • [12] Lamperti, J. (1963) Criteria for stochastic processes II: Passage-time moments. J. Math. Anal. Appl., 7, 127-145.
  • [13] Menshikov, M. and Popov, S. (1995) Exact power estimates for countable Markov chains. Markov Process. Related Fields, 1, 57-78.
  • [14] Meyn, S.P. and Tweedie, R.L. (1993) Markov Chains and Stochastic Stability. Berlin: Springer-Verlag.
  • [15] Nummelin, E. and Tuominnen, P. (1982) Geometric ergodicity of Harris recurrent Markov chains with applications to renewal theory. Stochastic Process. Appl., 12, 187-202.
  • [16] Nummelin, E. and Tweedie, R.L. (1978) Geometric ergodicity and R-positivity for general Markov chains. Ann. Probab., 6, 404-420.
  • [17] Nummelin, E. and Tweedie, R.L. (1994) Subgeometric rates of convergence of f-ergodic Markov chains. Adv. Appl. Probab., 26, 775-798.
  • [18] Tweedie, R.L. (1983) The existence of moments for stationary Markov chains. J. Appl. Probab., 28, 191-196.
  • [19] Williams, R.J. (1985) Recurrence classification for reflected Brownian motion in a wedge, Ann. Probab., 13, 758-778.