Open Access
December, 1955 On Transient Markov Processes with a Countable Number of States and Stationary Transition Probabilities
David Blackwell
Ann. Math. Statist. 26(4): 654-658 (December, 1955). DOI: 10.1214/aoms/1177728425


We consider a Markov process $x_0, x_1, \cdots$ with a countable set $S$ of states and stationary transition probabilities $p(t \mid s) = P\{x_{n+1} = t \mid x_n = s\}$. Call a set $C$ of states almost closed if (a) $P\{x_n \varepsilon C$ for an infinite number of $n\} > 0$ and (b) $x_n \varepsilon C$ infinitely often implies $x_n \varepsilon C$ for all sufficiently large $n$, with probability one. It is shown that there is a set $(C_1, C_2, \cdots)$ essentially unique, of disjoint almost closed sets such that (a) all except at most one of the $C_i$ are atomic, that is, $C_i$ does not contain two disjoint almost closed subsets, (b) the non-atomic $C_i$, if present, contains no atomic subsets, (c) the process is certain to enter and remain in some set $C_i$. A relation between the sets $C_i$ and the bounded solutions of the system of equations \begin{equation*}\tag{1} \alpha(s) = \sum_t \alpha(t)p(t \mid s)\end{equation*} is obtained; in particular there is only one atomic $C_i$ and no non-atomic $C_i$ if and only if the only bounded solution of (1) is $\alpha(t)$ = constant. This condition is shown to hold if the process is the sum of independent identical (numerical or vector) variables; whence, for such a process, the probability of entering a set $J$ infinitely often is zero or one. The results are new only if the process has transient components. The main tool is the martingale convergence theorem.


Download Citation

David Blackwell. "On Transient Markov Processes with a Countable Number of States and Stationary Transition Probabilities." Ann. Math. Statist. 26 (4) 654 - 658, December, 1955.


Published: December, 1955
First available in Project Euclid: 28 April 2007

zbMATH: 0066.11303
MathSciNet: MR75479
Digital Object Identifier: 10.1214/aoms/1177728425

Rights: Copyright © 1955 Institute of Mathematical Statistics

Vol.26 • No. 4 • December, 1955
Back to Top