The Annals of Applied Probability

State-Dependent Criteria for Convergence of Markov Chains

Sean P. Meyn and R. L. Tweedie

Full-text: Open access


The standard Foster-Lyapunov approach to establishing recurrence and ergodicity of Markov chains requires that the one-step mean drift of the chain be negative outside some appropriately finite set. Malyshev and Men'sikov developed a refinement of this approach for countable state space chains, allowing the drift to be negative after a number of steps depending on the starting state. We show that these countable space results are special cases of those in the wider context of $\varphi$-irreducible chains, and we give sample-path proofs natural for such chains which are rather more transparent than the original proofs of Malyshev and Men'sikov. We also develop an associated random-step approach giving similar conclusions. We further find state-dependent drift conditions sufficient to show that the chain is actually geometrically ergodic; that is, it has $n$-step transition probabilities which converge to their limits geometrically quickly. We apply these methods to a model of antibody activity and to a nonlinear threshold autoregressive model; they are also applicable to the analysis of complex queueing models.

Article information

Ann. Appl. Probab., Volume 4, Number 1 (1994), 149-168.

First available in Project Euclid: 19 April 2007

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier


Primary: 60J10: Markov chains (discrete-time Markov processes on discrete state spaces)

Foster's criterion irreducible Markov processes Lyapunov functions ergodicity geometric ergodicity recurrence Harris recurrence invasion models autoregressions networks of queues


Meyn, Sean P.; Tweedie, R. L. State-Dependent Criteria for Convergence of Markov Chains. Ann. Appl. Probab. 4 (1994), no. 1, 149--168. doi:10.1214/aoap/1177005204.

Export citation