Some Extensions of the Idea of Bias

Abstract

Laplace ([13], p. 44, lines 5 and 6), in his statement concerning the "milieu de probabilite", seems to have referred to a probability distribution of the true value of a certain quantity ("le veritable instant du phenomene"), or, as we would say at present, to a probability distribution of a certain parameter. Thereby he differs from the attitude adopted in most of the work discussed in the present paper. Yet, one might hold that he possessed the idea of median-unbiased estimators. At any rate, when applying his notions to what Todhunter ([26] p. 469, art. 875) calls a case of no practical value, Laplace ([13], p. 48, lines 11 and 12 from the bottom) virtually rejected the use of arithmetic means of observations. Judging from innumerable texts, one finds that after him emphasis has long been mainly on mean-unbiasedness (see, however, Pitman (]20], bottom of p. 215), who mentions the existence of bias in the sense that the probability that a certain mean-unbiased estimator is less than the parameter in question is $>\frac{1}{2}$). Yet it is hard to find the requirement of mean-unbiasedness justified in print (cf., Brown ([3], lines 6-8 of Section 3): the average of independent mean-unbiased estimates is consistent; Lehmann ([14], lines 4-10 from bottom of p. 588): mean-unbiasedness flows from his general concept in the case of a quadratic loss function; Birnbaum ([2], p. 32): mean-unbiasedness is merely a technically useful property of the classical estimators in the linear estimation problem, which, at least in the case of normal errors, could equally well or preferably be justified on the basis of median-unbiasedness), much harder, in fact, than to find warnings against the hope that much is gained if an estimator be mean-unbiased (cf., Kendall ([12], Vol. 2, Section 17.9); the examples provided by Girshick, Mosteller and Savage ([9], middle of p. 20), Halmos ([10], the end of p. 43), Savage ([23], bottom of p. 244); lack of invariance under certain transformations being stressed by Halmos ([10], bottom of p. 42), Brown ([3], lines 13-16 of Section 3), Fisher ([7], p. 143, line 13 from bottom)). All the same, much interesting work has been devoted to mean-unbiased estimators, some of it investigating the conversion of biased estimators into unbiased ones (e.g., Quenouille [21], Olkin and Pratt [17]), or deriving unbiased estimators ab initio (e.g. Tate [25]). It is not the purpose of this paper to provide a bibliography that is at all near completeness, but it is interesting that the last two references illustrate a statement, made by Schmetterer ([24], middle of p. 215), to the effect that a close connection exists between integral equations and linear operators on the one hand, and the theory of mean-unbiased estimators on the other. This suggests that part of the motivation for the research in this field is of a mathematical, rather than a statistical nature. This view seems to be corroborated by Fraser's statement ([8], lines 12-14 from bottom of p. 49) to the effect that median-unbiasedness does not seem to lend itself to the mathematical analysis needed to find minimum risk estimates, and hence has found little application. The present paper seeks to extend the notion of unbiasedness (and the notion of bias) in a direction different from Lehmann [14] (who gave a definition within the framework of general decision theory), and from Brown [3] (who was primarily concerned with types of unbiasedness, among them median-unbiasedness, that are invariant "under simultaneous one-to-one transformations of the parameter and (its) estimate", or rather under simultaneous strictly monotone transformations of the parameter and its estimate), and from Peterson's [19] density-unbiasedness. It originated in work by the author [29], [30], on the estimation of the latent roots of certain matrices occurring in response surface theory. It had become clear that in this case it was of primary interest whether or not the frequency of obtaining too small (or too large, respectively) estimates would be unduly large. The present paper will make this notion more precise. Several types of bias (or of unbiasedness, respectively) will emerge, all of them clearly invariant in the sense of Brown. Median-unbiasedness will turn out to be a special case of this larger concept. Finally, certain seemingly unfamiliar properties of the sample median, of the product-moment correlation coefficient, and of Olkin and Pratt's function of the latter [17] will be proved and used to illustrate some of the concepts discussed.