The Annals of Probability

Universal Almost Sure Data Compression

Donald S. Ornstein and Paul C. Shields

Full-text: Open access

Abstract

An $n$-code is a mapping $c_n$ from the set $A^n$ of sequences of length $n$ drawn from a finite set $A$ into the set of finite length binary sequences $B^\ast$. A decoder with distortion $D$ is a map from $B^\ast$ back into $A^n$ that sends $c_n(a^n_1)$ into a sequence that agrees with $a^n_1$ in all but at most $Dn$ places. We describe a sequence of codes and associated decoders of distortion $D$ such that, for almost every sequence from an ergodic process, the number of bits per $A$-symbol converges almost surely to $R(D)$, the optimal compression attainable for the process. The codes are universal in that the statistics of the process need not be known in advance. Expected value results of this type were first obtained by Davisson and, independently, Fittinghof; almost sure results for the invertible case $(D = 0)$ are implicitly contained in the Ziv-Lempel algorithm. Our results also apply, virtually without change in proof, to random fields.

Article information

Source
Ann. Probab., Volume 18, Number 2 (1990), 441-452.

Dates
First available in Project Euclid: 19 April 2007

Permanent link to this document
https://projecteuclid.org/euclid.aop/1176990840

Digital Object Identifier
doi:10.1214/aop/1176990840

Mathematical Reviews number (MathSciNet)
MR1055415

Zentralblatt MATH identifier
0699.94004

JSTOR
links.jstor.org

Subjects
Primary: 94A34: Rate-distortion theory
Secondary: 28D20: Entropy and other invariants

Keywords
Universal data compression entropy rate-distortion function

Citation

Ornstein, Donald S.; Shields, Paul C. Universal Almost Sure Data Compression. Ann. Probab. 18 (1990), no. 2, 441--452. doi:10.1214/aop/1176990840. https://projecteuclid.org/euclid.aop/1176990840


Export citation