A New Proof of the Pearson-Fisher Theorem

Abstract

This paper is concerned with the theorem that the $X^2$ goodness of fit statistic for a multinomial distribution with $r$ cells and with $s$ parameters fitted by the method of maximum likelihood is distributed as $\chi^2$ with $r - s - 1$ degrees of freedom. Karl Pearson formulated and proved the theorem for the special case $s = 0$. The general theorem was formulated by Fisher [2]. The first attempt at a rigorous proof is due to Cramer [1]. A serious weakness of Cramer's proof is that, in effect, he assumes that the maximum likelihood estimator is consistent. (To be precise, he proves the theorem for the subclass of maximum likelihood estimators that are consistent. But how are we in practice to distinguish between an inconsistent maximum likelihood estimator and a consistent one?) Rao [3] has closed this gap in Cramer's proof by proving the consistency of maximum likelihood for any family of discrete distributions under very general conditions. In this paper the theorem is proved under more general conditions than the combined conditions of Rao and Cramer. Cramer assumes the existence of continuous second partial derivatives with respect to the "unknown" parameter while here only total differentiability at the "true" parameter values is postulated. There is a radical difference in the method of proof. While Cramer regards the maximum likelihood estimate as being the point where the derivative of the log-likelihood function is zero, here it is regarded as the point at which the likelihood function takes values arbitrarily near to its supremum. The method of proof consists essentially of showing that the goodness of fit statistic is a quadratic form in the observed proportions when the observed proportions are close to the expected proportions. The known asymptotic properties of the multinomial distribution are then used. The asymptotic efficiency of the maximum likelihood estimator is proved at the same time.