The Annals of Mathematical Statistics

On Transient Markov Processes with a Countable Number of States and Stationary Transition Probabilities

David Blackwell

Full-text: Open access

Abstract

We consider a Markov process $x_0, x_1, \cdots$ with a countable set $S$ of states and stationary transition probabilities $p(t \mid s) = P\{x_{n+1} = t \mid x_n = s\}$. Call a set $C$ of states almost closed if (a) $P\{x_n \varepsilon C$ for an infinite number of $n\} > 0$ and (b) $x_n \varepsilon C$ infinitely often implies $x_n \varepsilon C$ for all sufficiently large $n$, with probability one. It is shown that there is a set $(C_1, C_2, \cdots)$ essentially unique, of disjoint almost closed sets such that (a) all except at most one of the $C_i$ are atomic, that is, $C_i$ does not contain two disjoint almost closed subsets, (b) the non-atomic $C_i$, if present, contains no atomic subsets, (c) the process is certain to enter and remain in some set $C_i$. A relation between the sets $C_i$ and the bounded solutions of the system of equations \begin{equation*}\tag{1} \alpha(s) = \sum_t \alpha(t)p(t \mid s)\end{equation*} is obtained; in particular there is only one atomic $C_i$ and no non-atomic $C_i$ if and only if the only bounded solution of (1) is $\alpha(t)$ = constant. This condition is shown to hold if the process is the sum of independent identical (numerical or vector) variables; whence, for such a process, the probability of entering a set $J$ infinitely often is zero or one. The results are new only if the process has transient components. The main tool is the martingale convergence theorem.

Article information

Source
Ann. Math. Statist., Volume 26, Number 4 (1955), 654-658.

Dates
First available in Project Euclid: 28 April 2007

Permanent link to this document
https://projecteuclid.org/euclid.aoms/1177728425

Digital Object Identifier
doi:10.1214/aoms/1177728425

Mathematical Reviews number (MathSciNet)
MR75479

Zentralblatt MATH identifier
0066.11303

JSTOR
links.jstor.org

Citation

Blackwell, David. On Transient Markov Processes with a Countable Number of States and Stationary Transition Probabilities. Ann. Math. Statist. 26 (1955), no. 4, 654--658. doi:10.1214/aoms/1177728425. https://projecteuclid.org/euclid.aoms/1177728425


Export citation