The Statistical Information Contained in Additional Observations

Abstract

Let $\mathscr{E}^n$ be a statistical experiment based on $n$ i.i.d. observations. We compare $\mathscr{E}^n$ with $\mathscr{E}^{n+r_n}$. The gain of information due to the $r_n$ additional observations is measured by the deficiency distance $\Delta (\mathscr{E}^n, \mathscr{E}^{n+r_n})$, i.e., the maximum diminution of the risk functions. We show that under general dimensionality conditions $\Delta(\mathscr{E}^n, \mathscr{E}^{n+r_n})$ is of order $r_n/n$. Further the behavior of $\Delta$ is studied and compared for asymptotically Gaussian experiments. We show that the information gain increases logarithmically. The Gaussian and the binomial family turn out to be--in some sense--opposite extreme cases, with the increase of information asymptotically minimal in the Gaussian case and maximal in the binomial.