This paper concerns the following question: if $X$ is a real-valued random variate having a one-parameter family of distributions $\mathscr{F}$, to what extent can $\mathscr{F}$ be normalized by a monotone transformation? In other words, does there exist a single transformation $Y = g(X)$ such that $Y$ has, nearly, a normal distribution for every distribution of $X$ in $\mathscr{F}$? The theory developed answers the question without considering the form of $g$ at all. In those cases where the answer is positive, simple formulas for calculating $g$ are given. The paper also considers the relationship between normalization and variance stabilization.

This paper discusses five questions concerning maximum likelihood estimation: What kind of theory is maximum likelihood? How is maximum likelihood used in practice? To what extent can this theory and practice be justified from a decision-theoretic viewpoint? What are maximum likelihood's principal virtues and defects? What improvements have been suggested by decision theory?

The differential-geometrical framework is given for analyzing statistical problems related to multi-parameter families of distributions. The dualistic structures of the exponential families and curved exponential families are elucidated from the geometrical viewpoint. The duality connected by the Legendre transformation is thus extended to include two kinds of affine connections and two kinds of curvatures. The second-order information loss is calculated for Fisher-efficient estimators, and is decomposed into the sum of two non-negative terms. One is related to the exponential curvature of the statistical model and the other is related to the mixture curvature of the estimator. Only the latter term depends on the estimator, and vanishes for the maximum-likelihood estimator. A set of statistics which recover the second-order information loss are given. The second-order efficiency also is obtained. The differential geometry of the function space of distributions is discussed.

Joint confidence and likelihood regions for the parameters in nonlinear regression models can be defined using the geometric concepts of sample space and solution locus. Using a quadratic approximation to the solution locus, instead of the usual linear approximation, it is shown that these inference regions correspond to ellipsoids on the tangent plane at the least squares point. Accurate approximate inference regions can be obtained by mapping these ellipsoids into the parameter space, and measures of the effect of intrinsic nonlinearity on inference can be based on the difference between the tangent plane ellipsoids and the sphere which would be obtained using a linear approximation.

A method is described for constructing sequential test boundaries for which the sample size distribution is easily computed. Assuming a diffusion approximation to be adequate, a few suitably placed images with negative weights are added to a basic unit source at the origin to create absorbing boundaries with intuitively desirable properties. A simple family of closed boundaries generated in this way is compared with Ferebee's (1980) reversed parabolic boundaries and Anderson's (1960) trapezoidal boundaries with regard to minimax expected sample size. The use of a delayed image to generate two-lobed closed boundaries is also briefly described.

Maximum likelihood estimation often fails when the parameter takes values in an infinite dimensional space. For example, the maximum likelihood method cannot be applied to the completely nonparametric estimation of a density function from an $\operatorname{iid}$ sample; the maximum of the likelihood is not attained by any density. In this example, as in many other examples, the parameter space (positive functions with area one) is too big. But the likelihood method can often be salvaged if we first maximize over a constrained subspace of the parameter space and then relax the constraint as the sample size grows. This is Grenander's "method of sieves." Application of the method sometimes leads to new estimators for familiar problems, or to a new motivation for an already well-studied technique. We will establish some general consistency results for the method, and then we will focus on three applications.

This paper concerns robust estimation of the parameter $\theta$ which indexes a parametric model for independent non-identically distributed data. For reasonable choices of contamination neighborhood and of what is to be estimated when the parametric model does not hold, we characterize asymptotically minimax robust estimates of $\theta$. When applied to the normal regression model, the theory yields recipes for the influence curves of optimal robust regression and scale estimates. The contamination neighborhood does not assume regression plus error structure, the regression and scale parameters are estimated simultaneously, and the theory establishes roles for estimates with redescending influence curves as well as for those with monotone influence curves. When applied to the logit and probit models, the theory recommends influence curves which differ markedly from those of the maximum likelihood estimates except in the i.i.d. case.

We consider a heteroscedastic linear model in which the variances are given by a parametric function of the mean responses and a parameter $\theta$. We propose robust estimates for the regression parameter $\beta$ and show that, as long as a reasonable starting estimate of $\theta$ is available, our estimates of $\beta$ are asymptotically equivalent to the natural estimate obtained with known variances. A particular method for estimating $\theta$ is proposed and shown by Monte-Carlo to work quite well, especially in power and exponential models for the variances. We also briefly discuss a "feedback" estimate of $\beta$.

Stochastic approximation methods are used to generate a sequence of "$M$-estimates" for the unknown parameters of an autoregressive process of known, finite order which may have heavy-tailed innovations. Weak dependence properties, which can be demonstrated for many autoregressive processes, are used in the proof that the sequence converges almost surely to the parameters. A brief Monte Carlo study verifies that bounded influence functions provide protection for recursive procedures against heavy-tailed innovations.

If $M$ is not convex, and the underlying density is multi-modal, $M$-estimators can be inconsistent. Examples are given, as well as some positive results.

Often a statistic of interest would take the form of a member of a common family, except that some vital parameter is unknown and must be estimated. This paper describes methods for showing the asymptotic normality of such statistics with estimated parameters. Whether or not the limiting distribution is affected by the estimator is primarily a question of whether or not the limiting mean (derived by replacing the estimator by a mathematical variable) has a nonzero derivative with respect to that variable. Section 2 contains conditions yielding the asymptotic normality of $U$-statistics with estimated parameters. These results generalize previous theorems by Sukhatme (1958). As an example, we show the limiting normality of a resubstitution estimator of a correct classification probability when using Fisher's linear discriminant function. The results for $U$-statistics are extended to cover a broad class of families of statistics through the differential. Specifically, conditions are given which yield the asymptotic normality of adaptive $L$-statistics and an example due to de Wet and van Wyk (1979) is examined.

In a variety of statistical problems, one is interested in the limiting distribution of statistics $\hat{T}_n = T_n(y_1, y_2, \cdots, y_n; \hat{\lambda}_n)$, where $\hat{\lambda}_n$ is an estimator of a parameter in the distribution of the $y_i$ and where the limiting distribution of $T_n = T_n(y_1, y_2, \cdots, y_n; \lambda)$ is relatively easy to find. For cases in which the limiting distribution of $T_n$ is normal with mean independent of $\lambda$, a useful method is given for finding the limiting distribution of $\hat{T}_n$. A simple application to testing normality in regression models is given.

Arbitrary nonparametric mixtures of exponential and Weibull (fixed shape) distributions are considered as possible models for a lifetime distribution. A characterization of such distributions is given by the well-known characterization of Laplace transforms. The maximum likelihood estimate of the mixing distribution is investigated and found to be supported on a finite number of points. It is shown to be unique and weakly convergent to the true mixing measure with probability one. A practical algorithm for computing the maximum likelihood estimate is described. Its performance is briefly discussed and some illustrative examples given.

Variance and bias comparisons are set forth for a sequence of nonlinear statistics based on independent random variables including jackknife and U-statistics of various orders. The analysis relies heavily on an orthogonal decomposition first introduced by Hoeffding in 1948. This ANOVA type decomposition is refined for purposes of discerning higher order convexity properties for an array of conditional variance coefficients. There is also some discussion of two-sample statistics.

In the setting where the weights of $k$ objects are to be determined in $n$ weighings on a chemical balance (or equivalent problems), for $n \equiv 3 (\operatorname{mod} 4)$, Ehlich and others have characterized certain "block matrices" $C$ such that, if $X'X = C$ where $X(n \times k)$ has entries $\pm1$, then $X$ is an optimum design for the weighing problem. We give methods here for constructing $X$'s for which $X'X$ is a block matrix, and show that it is the optimum $C$ for infinitely many $(n, k)$. A table of known constructibility results for $n < 100$ is given.

The model $E(y \mid x) = \theta_0 + \sum^k_{i=1} \theta_ix_i + \psi(\mathbf{x})$ is considered, where $\psi(\mathbf{x})$ is an unknown contamination with $| \psi(\mathbf{x})|$ bounded by given $\varphi(\mathbf{x})$. Optimal designs are studied in terms of least squares estimation and a family of minimax criteria. In particular, analogs of D-, A- and E-optimal designs are studied in the general case of an arbitrary $k$. Some commonly used integer designs are considered and their efficiencies with respect to optimal designs are determined. In particular, it is shown that star-point designs or regular replicas of $2^k$ factorials are very efficient under the appropriate choice of levels of factors.

The integral of a second-order stochastic process $Z$ over a $d$-dimensional domain is estimated by a weighted linear combination of observations of $Z$ in a random design. The design sample points are possibly dependent random variables and are independent of the process $Z$, which may be nonstationary. Necessary and sufficient conditions are obtained for the mean squared error of a random design estimator to converge to zero as the sample size increases towards infinity. Simple random, stratified and systematic sampling designs are considered; an optimal simple random design is obtained for fixed sample size; and the mean squared errors of the estimators from these designs are compared. It is shown, for example, that for any simple random design there is always a better stratified design.

An estimation procedure is presented for the coefficients of the nonlinear functional relation, where observations are subject to measurement error. The distributional properties of the estimators are derived, and a consistent estimator of the covariance matrix is given. In deriving the results it is assumed that the covariance matrix of the observational errors is known and that this covariance matrix is $o(n^{-1/3})$, where $n$ is the index of the sequence of estimators.

This paper describes how sufficiency and invariance considerations can be applied in problems of confidence set estimation to reduce the class of set estimators under investigation. Let $X$ be a random variable taking values in $\mathscr{X}$ with distribution $P_\theta, \theta \in \Theta$, and suppose a confidence set is desired for $\gamma = \gamma(\theta)$, where $\gamma$ takes values in $\Gamma$. The main tools used are (i) the representation of randomized set estimators as functions $\varphi: \mathscr{X} \times \Gamma \rightarrow \lbrack 0,1 \rbrack$, and (ii) the definition of sufficiency in terms of a certain family of distributions on $\mathscr{X} \times \Gamma$. Sufficiency and invariance reductions applied in tandem to $\mathscr{X} \times \Gamma$ yield a class of set estimators that is essentially complete among all invariant set estimators, provided the risk function depends only on $E_{\theta \varphi} (X, \gamma), (\theta, \gamma) \in \Theta \times \Gamma$. Several illustrations are given.

We propose a method of estimating the endpoint, $\theta$, of a distribution when only limited information is available about the behaviour of the distribution in the neighbourhood of $\theta$. By using increasing numbers of extreme order statistics we obtain an estimator which improves on earlier estimators based on only a bounded number of extremes. In a certain particular model our estimator is equal to a maximum likelihood estimator, but it is robust against departures from this model.

The two-sample problem of testing against location shift is fundamental to much of the theory of rank tests. Generally testing and estimation is carried out with a fixed (non-random) set of scores for the ranks. However Beran (1974), following ideas of Stein and Hajek, developed a notable class of adaptive estimators. When used in testing, these give asymptotically efficient tests, regardless of the underlying distribution. These ideas are used here to focus attention upon tests generated by continuous, piecewise linear functions (called PLRT's) which provide a practically useful class of asymptotically efficient adaptive rank tests. Under suitable conditions the rate of convergence of the consistent estimators of the score generating function is $O(N^{-1/2})$ which suggests they are quite suitable for practical application when $N$ is large. A Riesz representation theorem for the asymptotic power of linear rank tests is obtained which amongst other things permits the derivation of optimal PLRT's under weaker conditions than are required for optimal linear rank tests. Further useful properties of PLRT's are noted.

The concept of $H_0$-contiguity is studied for certain nonparametric testing problems. Furthermore, it is shown that a meaningful sample definition of Pitman efficiency is possible under $H_0$-contiguity, and this definition turns out to coincide with the usual efficiency concept based on comparing asymptotic distributions.

In 1952 Chernoff introduced a measure of asymptotic efficiency for tests. Comparison in the sense of Chernoff is concerned with fixed alternatives. In contrast to Bahadur's approach, where the probabilities of first and second kind are treated in an unbalanced way, in Chernoff's approach both probabilities go to zero. For the calculation of Chernoff efficiencies one has to develop large deviation theorems both under the null hypothesis and under the alternative hypothesis. In this paper some basic properties are mentioned and the concept of Chernoff deficiency is introduced in a manner analogous to the Pitman and Bahadur case. It is shown that in typical testing problems in multivariate exponential families, the likelihood ratio test is Chernoff deficient of order $\mathscr{O}(\log n)$. Many of the results agree with corresponding results in the Bahadur case.

The product limit estimator due to Kaplan and Meier (1958) is well-known to be the nonparametric maximum likelihood estimator of a distribution function based on censored data. It is shown here that the product limit estimator is an asymptotically optimal estimator in two senses: in the sense of a Hajek-Beran type representation theorem for regular estimators; and in an asymptotic minimax sense similar to the classical result for the uncensored case due to Dvoretzky, Kiefer, and Wolfowitz (1956). The proofs rely on the methods of Beran (1977) and Millar (1979).

This paper is concerned with the theory of unbiased estimation for nonparametric families of distributions subject to "generalised moment" restrictions. Necessary and sufficient conditions under which symmetric statistics are unique minimum variance unbiased estimators of their expectations are obtained, and some new boundedly complete families of distributions are exhibited.

We derive the nonparametric maximum likelihood estimate, $\hat{F}$ say, of a lifetime distribution $F$ on the basis of two independent samples, one a sample of size $m$ from $F$ and the other a sample of size $n$ from the length-biased distribution of $F$, i.e. from $G_F(x) = \int^x_0 u dF(u)/\mu, \mu = \int^\infty_0 x dF(x)$. We further show that $(m + n)^{1/2}(\hat{F} - F)$ converges weakly to a pinned Gaussian process with a simple covariance function, when $m + n \rightarrow \infty$ and $m/n \rightarrow$ constant. Potential applications are described.

The asymptotic properties of a family of minimum quantile function distance estimators are considered. These procedures take as the parameter estimates that vector which minimizes a weighted $L^2$ distance between the empirical quantile function and an assumed parametric family of quantile functions. Regularity conditions needed for these estimators to be consistent and asymptotically normal are presented. For single parameter families of distributions, the optimal form of the weight function is presented.

For any decision problem, one wishes to find that estimator which minimizes the expected loss. If the loss function is squared error, then the estimator is the mean of the Bayes posterior distribution. Unfortunately the prior distribution may be unknown, but in certain situations empirical Bayes methods can circumvent this problem by using past observations to estimate either the prior or the Bayes estimate directly. Empirical Bayes methods are particularly appealing when the Bayes estimate depends only on the marginal distribution of the observed variable, yielding what is known as a simple empirical Bayes estimate. The paper looks at the underlying circumstance of when a simple empirical Bayes estimator is available, and shows its occurrence not to be happenstance.

If a strictly stationary process $\{Z_k\}$ has residuals $Z_{k+1} - \sum^k_{j=1} a_{k,j}Z_j$ independent of $(Z_1, \cdots, Z_k)$ for all $k \geq m$, it is shown that the process is Gaussian or degenerate or $m$-step Markovian. Generalized (nonlinear) autoregressive stationary processes are defined and partially characterized.

An urn contains a known number of balls, an unknown number $R$ of which are red. Sequential sampling with replacement is possible and cost is proportional to sample size. The objective is to estimate $R$ with 0-1 loss, given that a priori $R$ has a discrete uniform distribution. It is shown that optimal stopping regions may be disconnected and composed of islands and peninsulas.

It is shown that if the density $f(\mathbf{x})$ of $\mathbf{X} = (X_1, \cdots, X_n)$ is Schur-concave, then (1) $P(|X_i| \leq a_i, i = 1, \cdots, n)$ is a Schur-concave function of $(\phi(a_1), \cdots, \phi(a_n))$, and (2) $P\{\Sigma(X_i/a_i)^2 \leq 1\}$ is a Schur-concave function of $(\phi(a^2_1), \cdots, \phi(a^2_n))$, where $\phi; \lbrack 0, \infty) \rightarrow \lbrack 0, \infty)$ is any increasing and convex function. By letting $\phi(a) = a$, (1) implies that $P(|X_i| \leq a_i, i = 1, \cdots, n) \leq P(|X_i| \leq \bar{a}, i = 1, \cdots, n)$. As special consequences, the results yield bounds for exchangeable normal and $t$ variables and for linear combinations of central and noncentral Chi squared variables.

Let $\{X_i\}$ be a sample from $F(x - \theta)$ where $F$ is in a class $\mathscr{F}$ of symmetric distributions on the line and $\theta$ is the location parameter to be estimated. Huber has shown that maximum likelihood estimation has a minimax property over a convex $\mathscr{F}$. Here a simple convex $\mathscr{F}$ is given for which neither $L$- nor $R$-estimation has the minimax property. In particular, this example shows that a recent assertion concerning $L$-estimation is not true.