Estimation of a Convex Real Parameter of an Unknown Information Source

Abstract

Let $\mathscr{P}$ be the family of all stationary information sources with alphabet $A$. Let $F: \mathscr{P} \rightarrow(-\infty, \infty)$ be convex and upper semicontinuous in the weak topology. It is shown that for $n = 1,2, \cdots$, there is an estimator $Y_n: A^n \rightarrow (-\infty, \infty)$, such that if $\mu \in \mathscr{P}$ is ergodic and the process $(X_1, X_2,\cdots)$ has distribution $\mu$, then $Y_n(X_1,\cdots, X_n)\rightarrow F(\mu)$ in $L^1$ mean.