December, 1955 On Transient Markov Processes with a Countable Number of States and Stationary Transition Probabilities
David Blackwell
Ann. Math. Statist. 26(4): 654-658 (December, 1955). DOI: 10.1214/aoms/1177728425

## Abstract

We consider a Markov process $x_0, x_1, \cdots$ with a countable set $S$ of states and stationary transition probabilities $p(t \mid s) = P\{x_{n+1} = t \mid x_n = s\}$. Call a set $C$ of states almost closed if (a) $P\{x_n \varepsilon C$ for an infinite number of $n\} > 0$ and (b) $x_n \varepsilon C$ infinitely often implies $x_n \varepsilon C$ for all sufficiently large $n$, with probability one. It is shown that there is a set $(C_1, C_2, \cdots)$ essentially unique, of disjoint almost closed sets such that (a) all except at most one of the $C_i$ are atomic, that is, $C_i$ does not contain two disjoint almost closed subsets, (b) the non-atomic $C_i$, if present, contains no atomic subsets, (c) the process is certain to enter and remain in some set $C_i$. A relation between the sets $C_i$ and the bounded solutions of the system of equations \begin{equation*}\tag{1} \alpha(s) = \sum_t \alpha(t)p(t \mid s)\end{equation*} is obtained; in particular there is only one atomic $C_i$ and no non-atomic $C_i$ if and only if the only bounded solution of (1) is $\alpha(t)$ = constant. This condition is shown to hold if the process is the sum of independent identical (numerical or vector) variables; whence, for such a process, the probability of entering a set $J$ infinitely often is zero or one. The results are new only if the process has transient components. The main tool is the martingale convergence theorem.

## Citation

Download Citation

David Blackwell. "On Transient Markov Processes with a Countable Number of States and Stationary Transition Probabilities." Ann. Math. Statist. 26 (4) 654 - 658, December, 1955. https://doi.org/10.1214/aoms/1177728425

## Information

Published: December, 1955
First available in Project Euclid: 28 April 2007

zbMATH: 0066.11303
MathSciNet: MR75479
Digital Object Identifier: 10.1214/aoms/1177728425

Rights: Copyright © 1955 Institute of Mathematical Statistics

JOURNAL ARTICLE
5 PAGES

Vol.26 • No. 4 • December, 1955