April, 1967 A Potential Theoretic Proof of a Theorem of Derman and Veinott
Ronald A. Schaufele
Ann. Math. Statist. 38(2): 585-587 (April, 1967). DOI: 10.1214/aoms/1177698974

## Abstract

The purpose of this note is to provide alternative proofs of results of Derman and Veinott . The method of proof uses the potential theory for Markov chains developed by Kemeny and Snell in , , , . In , Kemeny and Snell treat transient and recurrent chains separately, whereas Derman and Veinott consider chains having one positive recurrent class, $C, (0 \epsilon C)$, and a set of transient states, $T$, with $T \cup C = \{0, 1, 2, \cdots\}$. This difference does not cause any great difficulty. Kemeny and Snell also assume in  that their recurrent chains are noncyclic. The recurrent class, $C$, will therefore be assumed noncyclic in this note, the extension to the cyclic case being handled in the usual way. By relabelling the states, one may write the transition matrix, $P$, as \begin{equation*}\tag{1.1}P = \begin{pmatrix}P^1 R 0 \\ R Q\end{pmatrix}\end{equation*} where $P^1$ is the transition matrix of the recurrent class and $Q$ is the (substochastic) transition matrix of the set of transient states. In this note, the notation will be that of Derman and Veinott  and all references to the work of Kemeny and Snell will be to . In matrix notation, the Derman-Veinott equation is \begin{equation*}\tag{1.2} (I - P)v = \omega^{\ast}\end{equation*} where $\omega{^\ast} = \omega - g\mathbf{1}, \omega = (\omega_0, \omega_1, \omega_2, \cdots)^T$ is a known vector, $\mathbf{1} = (1, 1, 1, \cdots)^T, v = (v_0, v_1, v_2, \cdots)^T$ is an unknown vector and $g$ is an unknown constant.

## Citation

Ronald A. Schaufele. "A Potential Theoretic Proof of a Theorem of Derman and Veinott." Ann. Math. Statist. 38 (2) 585 - 587, April, 1967. https://doi.org/10.1214/aoms/1177698974

## Information

Published: April, 1967
First available in Project Euclid: 27 April 2007

zbMATH: 0171.16005
MathSciNet: MR208772
Digital Object Identifier: 10.1214/aoms/1177698974 