On the Estimation of Parameters Restricted by Inequalities

Abstract

There are collected in this paper several observations and results more or less loosely related by their connections with the subject mentioned in the title. The discussion moves from the general to the specific, beginning with some remarks on minimization of convex functions subject to side conditions, and ending with a discussion of uniform consistency of estimators of linearly ordered parameters. Section 2 deals with one aspect of the problem of minimizing a function of several variables, subject to side conditions which specify that the variables must satisfy certain inequalities. It is frequently true in such problems that information as to which of the restricting sets contain the minimizing point on their boundaries is of great assistance in finding this point. Theorem 2.1 provides the basis for a stepwise procedure leading to this information when both the function to be minimized and the restricting sets are convex. It makes no contribution, however, to the problem of finding the minimizing point on a given boundary or intersection of boundaries. Brief mention is made in Section 3 of some examples of estimation problems for which the remark to which Section 2 is devoted is appropriate. Section 4 is concerned with a situation in which samples are taken from $k$ populations, each known to belong to a given one-parameter "exponential family". The problem is the maximum likelihood estimation of the $k$ parameters determining the populations, subject to certain restrictions. Methods are discussed of finding the minimizing point on a given intersection of boundaries of restricting sets. In the particular case when all populations belong to the same exponential family and when the restrictions on the parameters are order restrictions, it is observed that the maximum likelihood estimators (MLE's) of the means are independent of the particular exponential family. In Section 5 is discussed a property, related to sufficiency, of the MLE's discussed in Section 4. Let $y$ denote a vector representing a set of possible values of the MLE's, $E$ a Borel subset of the sample space, $\tau$ a parameter point, $S_0$ the intersection of the restricting sets. If $S_0$ is bounded by hyperplanes, there is a determination of the conditional probability $p\tau(E \mid y)$ which is independent of $\tau$ when $y$ is interior to $S_0$, and, when $y$ lies on a face, edge, or vertex of $S_0$, is independent of $\tau$ on the closure of that face, edge, or vertex. This result may be regarded as a generalization of a remark ([16], p. 77) to the effect that if $X$ and $Y$ are normally distributed random variables with unit standard deviation and means $\xi$ and $\eta$ respectively, and if $\xi$ and $\eta$ are known to satisfy a linear equation, then the foot of the perpendicular from the observation point $(x, y)$ to the line is a sufficient estimator. Section 6 is devoted to the same problem as are Sections 4 and 5, except that the parameters are linearly ordered, and that the populations need not belong to exponential families. Conditions are obtained for the strong uniform consistency of an estimator which is the MLE when the populations do belong to the same exponential family. An asymptotic lower bound is given for the probability of achieving a given precision uniformly.