Measures of Information Based on Comparison with Total Information and with Total Ignorance

Abstract

Two measures of content of information in statistical experiments are considered. They are both based on Le Cam's notion of deficiency of one experiment with respect to another. The measures are: (i) The deficiency of the given experiment with respect to a totally informative experiment. (ii) The deficiency of a totally uninformative experiment with respect to the given experiment. We shall here discuss the interpretations of such measures, establish inequalities for them and related quantities, and study their behaviour under replications. If the parameter set is finite then closed expressions for exponential rates of convergence, as the number of replications increase, are given. In particular the exponential rate of the minimax probability of not covering the true values of the parameter by an $r$-point confidence set is expressed in terms of Hellinger transforms. If convergence to the totally informative experiment takes place at all, then the speed of convergence is necessary exponential. Examples are given indicating various possibilities.