Relative Entropy Densities and a Class of Limit Theorems of the Sequence of $m$-Valued Random Variables

Abstract

Let $\{X_n, n \geq 1\}$ be a sequence of random variables taking values in $S = \{1,2, \ldots, m\}$ with distribution $p(x_1, \ldots, x_n), (p_{i1}, p_{i2}, \ldots, p_{im}), i = 1,2, \ldots$, a sequence of probability distributions on $S$, and $\varphi_n = (1/n)\log p(X_1, \ldots, X_n) - (1/n)\sum^n_{i = 1}\log p_{iX_i}$ the entropy density deviation, relative to the distribution $\prod^n_{i = 1}p_{ix_i}, \text{of} \{X_i, 1 \leq i \leq n\}$. In this paper the relation between the limit property of $\varphi_n$ and the frequency of given values in $\{X_n\}$ is studied.