Open Access
April, 1988 A Sandwich Proof of the Shannon-McMillan-Breiman Theorem
Paul H. Algoet, Thomas M. Cover
Ann. Probab. 16(2): 899-909 (April, 1988). DOI: 10.1214/aop/1176991794


Let $\{X_t\}$ be a stationary ergodic process with distribution $P$ admitting densities $p(x_0,\ldots, x_{n-1})$ relative to a reference measure $M$ that is finite order Markov with stationary transition kernel. Let $I_M(P)$ denote the relative entropy rate. Then $n^{-1}\log p(X_0,\ldots, X_{n-1}) \rightarrow I_M(P) \mathrm{a.s.} (P).$ We present an elementary proof of the Shannon-McMillan-Breiman theorem and the preceding generalization, obviating the need to verify integrability conditions and also covering the case $I_M(P) = \infty$. A sandwich argument reduces the proof to direct applications of the ergodic theorem.


Download Citation

Paul H. Algoet. Thomas M. Cover. "A Sandwich Proof of the Shannon-McMillan-Breiman Theorem." Ann. Probab. 16 (2) 899 - 909, April, 1988.


Published: April, 1988
First available in Project Euclid: 19 April 2007

zbMATH: 0653.28013
MathSciNet: MR929085
Digital Object Identifier: 10.1214/aop/1176991794

Primary: 28D05
Secondary: 28A65 , 28D20 , 60F15 , 94A17

Keywords: asymptotic equipartition property (AEP) , asymptotically mean stationary , ergodic theorem of information theory , likelihood ratio , Markov approximation , relative entropy rate , sandwich argument , Shannon-McMillan-Breiman theorem

Rights: Copyright © 1988 Institute of Mathematical Statistics

Vol.16 • No. 2 • April, 1988
Back to Top