A Simple Method for Improving some Estimators

Abstract

In the past, the principles which have been applied most often in the selection of an estimate are the principles of maximum likelihood and of minimum variance unbiased estimation. Recent statistical literature (e.g., [1]) has pointed out the fact that, while these principles are intuitively appealing, neither of them can be justified very well in a systematic development of statistics. Abraham Wald [2] has indicated a more systematic approach to the problem. One of Wald's ideas may be paraphrased as follows. Consider a random variable $X$ whose distribution depends on an unknown real parameter $\theta$. If the value $x$ of $X$ is observed one makes an estimate, say $f(x)$, and thereby incurs a loss of $W\lbrack\theta,f(x)\rbrack$. The risk associated with the estimate $f$ is defined to be the expected loss $R_f(\theta) = E\{W\lbrack\theta, f(X)\rbrack\mid\theta\}$. In choosing between two estimators $f_1$ and $f_2$, it seems clear that one would prefer $f_1$ to $f_2$ if $R_{f_1}(\theta) \leqq R_{f_2}(\theta)$ for all values of $\theta$, and $R_{f_1}(\theta) < R_{f_2}(\theta)$ for at least one value of $\theta$. We shall consider only the case where the loss as a function of $\theta$ and the estimate $f(x)$ is of the special form \begin{equation*}\tag{1}W\lbrack\theta,f(x)\rbrack = \lambda(\theta)(f - \theta)^2;\lambda(\theta) > 0.\end{equation*} Reasons for considering this form of $W\lbrack\theta,f(x)\rbrack$ have been given in [3]. Suppose that we know of an unbiased estimate $f$ whose variance is $K\theta^2$, where $K$ is known. Then, as we shall see, the risk of $f$ is greater than the risk of $f/(K + 1)$. Hence $f/(K + 1)$ is to be preferred to $f$ as an estimator of $\theta$. This result holds for any function $\lambda(\theta) > 0$. Although $f/(K + 1)$ is generally not unbiased in the usual sense, it is unbiased in a certain sense (cf. [4]). It is seen that a special case of this result is related to the problem of the estimation of the scale parameter of a population whose form is not given but for which the ratio of the first and second moments is known (cf. [5]). Some special cases and applications are discussed in detail.