On the Information Contained in Additional Observations

Abstract

Let $\{X_j; j = 1, 2, \cdots\}$ be independent identically distributed random variables whose individual distribution $p_\theta$ is indexed by a parameter $\theta$ in a set $\Theta$. For two integers $m < n$ the experiment $\mathscr{E}_n$ which consists in observing the first $n$ variables is more informative than $\mathscr{E}_m$. Two measures of the supplementary information are described. One is the deficiency $\delta (\mathscr{E}_m, \mathscr{E}_n)$ introduced by this author. Another is a number $\eta(\mathscr{E}_m, \mathscr{E}_n)$ called "insufficiency" and related to previous arguments of Wald (1943). Relations between $\delta$ and $\eta$ are described. One defines a dimensionality coefficient $D$ for $\Theta$ and obtains a bound of the type $\eta(\mathscr{E}_m, \mathscr{E}_n) \leqq \lbrack 2D(n - m)/n\rbrack^{\frac{1}{2}}.$ Examples show that $\delta(\mathscr{E}_m, \mathscr{E}_n)$ may stay bounded away from zero in infinite dimensional cases, even if $m \rightarrow \infty$ and $n = m + 1$.