On Transient Markov Chains with Application to the Uniqueness Problem for Markov Processes

Abstract

We focus our attention herein on a Markov chain $x_0, x_1, \cdots$ with a countable number of states indexed by a subset I of the integers and with stationary transition probabilities $p_{ij}$, and explore the sets of states defined by: A transient set of states $C$ is said to be denumerably atomic if $P(x_n \varepsilon C i.o.) > 0$ and if for every infinite set $A \subset C$ we have $x_n \varepsilon C i.o.$ implies $x_n \varepsilon A i.o.$ with probability one (a.s.). Following Blackwell's basic paper [1] which introduced the systematic use of martingales into the study of Markov chains, we use the semi-martingale convergence theorem [2] to characterize denumerably atomic sets in terms of the bounded solutions of the inequality $$\phi(i) \leqq \sum_{j \varepsilon I} p_{ij}\phi(j),\quad i \varepsilon I.$$ For chains whose state space contains a denumerably atomic set a convergence criterion for certain sums $\sum^\infty_{n = 0}f(x_n)$ is then developed. The application of this criterion to a restricted class of continuous parameter Markov processes gives simple necessary and sufficient conditions for the existence of a unique process satisfying given infinitesimal conditions. This last result illuminates the connection between the necessary and sufficient conditions given by Feller [3] for uniqueness and the simpler conditions for birth and death processes given recently by Dobrusin [4], more recently by Karlin and McGregor [5], and by Reuter and Lederman [6] (see also [7]).