### A Sandwich Proof of the Shannon-McMillan-Breiman Theorem

Paul H. Algoet and Thomas M. Cover
Source: Ann. Probab. Volume 16, Number 2 (1988), 899-909.

#### Abstract

Let $\{X_t\}$ be a stationary ergodic process with distribution $P$ admitting densities $p(x_0,\ldots, x_{n-1})$ relative to a reference measure $M$ that is finite order Markov with stationary transition kernel. Let $I_M(P)$ denote the relative entropy rate. Then $n^{-1}\log p(X_0,\ldots, X_{n-1}) \rightarrow I_M(P) \mathrm{a.s.} (P).$ We present an elementary proof of the Shannon-McMillan-Breiman theorem and the preceding generalization, obviating the need to verify integrability conditions and also covering the case $I_M(P) = \infty$. A sandwich argument reduces the proof to direct applications of the ergodic theorem.

First Page:
Primary Subjects: 28D05
Secondary Subjects: 94A17, 28A65, 28D20, 60F15
Full-text: Open access