## The Annals of Probability

- Ann. Probab.
- Volume 16, Number 2 (1988), 899-909.

### A Sandwich Proof of the Shannon-McMillan-Breiman Theorem

Paul H. Algoet and Thomas M. Cover

#### Abstract

Let $\{X_t\}$ be a stationary ergodic process with distribution $P$ admitting densities $p(x_0,\ldots, x_{n-1})$ relative to a reference measure $M$ that is finite order Markov with stationary transition kernel. Let $I_M(P)$ denote the relative entropy rate. Then $n^{-1}\log p(X_0,\ldots, X_{n-1}) \rightarrow I_M(P) \mathrm{a.s.} (P).$ We present an elementary proof of the Shannon-McMillan-Breiman theorem and the preceding generalization, obviating the need to verify integrability conditions and also covering the case $I_M(P) = \infty$. A sandwich argument reduces the proof to direct applications of the ergodic theorem.

#### Article information

**Source**

Ann. Probab., Volume 16, Number 2 (1988), 899-909.

**Dates**

First available in Project Euclid: 19 April 2007

**Permanent link to this document**

https://projecteuclid.org/euclid.aop/1176991794

**Digital Object Identifier**

doi:10.1214/aop/1176991794

**Mathematical Reviews number (MathSciNet)**

MR929085

**Zentralblatt MATH identifier**

0653.28013

**JSTOR**

links.jstor.org

**Subjects**

Primary: 28D05: Measure-preserving transformations

Secondary: 94A17: Measures of information, entropy 28A65 28D20: Entropy and other invariants 60F15: Strong theorems

**Keywords**

Shannon-McMillan-Breiman theorem asymptotic equipartition property (AEP) ergodic theorem of information theory relative entropy rate likelihood ratio sandwich argument Markov approximation asymptotically mean stationary

#### Citation

Algoet, Paul H.; Cover, Thomas M. A Sandwich Proof of the Shannon-McMillan-Breiman Theorem. Ann. Probab. 16 (1988), no. 2, 899--909. doi:10.1214/aop/1176991794. https://projecteuclid.org/euclid.aop/1176991794