We simplify a general heuristic necessary condition of Stein's for adaptive estimation of a Euclidean parameter in the presence of an infinite dimensional shape nuisance parameter and other Euclidean nuisance parameters. We derive sufficient conditions and apply them in the construction of adaptive estimates for the parameters of linear models and multivariate elliptic distributions. We conclude with a review of issues in adaptive estimation.

Asymptotic expansions are derived for the densities of multivariate $M$-estimates. The expansion is based on a saddlepoint technique and yields good accuracy in the tails for small sample sizes. Numerical results are given for robust estimation of location and scale and these are compared with known Monte Carlo results.

For each $t$ in an index set $T$, let $P_t$ be a probability measure with mode $M(t)$. In this paper we consider a maximum likelihood nonparametric estimator $\hat{M}_n(t)$ of $M(t)$ subject to the constraint that $\hat{M}_n(\cdot)$ be isotonic with respect to an order on $T$. The estimator is a solution to a minimization problem with zero-one loss. The estimator is not a max-min or min-max representation of "naive" estimators. Naive modal estimators are used but they are not linear in the sense of Robertson and Wright (1975) nor do they have the Cauchy mean value property. Consistency results are given for the cases of $T$ finite, $P_t$ discrete; $T$ infinite, $P_t$ continuous; $T$ finite, $P_t$ continuous. An efficient and economical quadratic-time dynamic programming algorithm is presented for computations. In the case of $T$ finite, $P_t$ discrete, the algorithm operates on a matrix of frequency counts, "backing up" one column at a time as the optimal completion cells for the modal estimates are searched for. Illustrative simulations suggest that the estimator performs well in small samples and is robust to certain kinds of contamination perturbations.

Algorithms for solving the isotonic regression problem in more than one dimension are difficult to implement because of the large number of lower sets present or because they involve search techniques which require a significant amount of checking and readjustment. Here a new algorithm for solving this problem based on a simple iterative technique is proposed and shown to converge to the correct solution.

Let $\Theta$ be a convex subset of $R^d$, and for each $\theta \in \Theta$ let $F(\theta; dt)$ be a probability on the line. For any vector $a_n = (a_{n1}, a_{n2}, \cdots, a_{nn})$ where $a_{ni} \in \Theta$, let $X_{n1}, \cdots, X_{nn}$ be independent observations, the distribution of $X_{ni}$ being $F(a_{ni}; dt)$. The main results give a method of estimating the unknown regression function $a_n$ based on a minimum distance recipe. Under regularity assumptions, the proposed estimators are shown, in an appropriate framework, to be asymptotically normal, locally asymptotically minimax, and robust. The abstract results are illustrated by application to the linear model and to exponential response models. In general, nothing at all is assumed about the form of the regression function; accordingly, this forces the limiting normal distributions of the proposed estimators to be located on infinite dimensional linear spaces.

We present elements of a frequentist theory of statistics for concepts of upper and lower (interval-valued) probability (IVP), defined on finite event algebras. We consider IID models for unlinked repetitions of experiments described by IVP and suggest several generalizations of standard notions of independence, asymptotic certainty and estimability. Instability of relative freqencies is favoured under our IID models. Moreover, generalizations of Bernoulli's Theorem give some justification for the estimation of an underlying IVP mechanism from fluctuations of relative frequencies. Our results indicate that an objectivist, frequency- or propensity-oriented, view of probability does not necessitate an additive probability concept, and that IVP models can represent a type of indeterminacy not captured by additive probability.

In this paper the exponential rates of decrease and bounds on tail probabilities for consistent estimators are studied using large deviation methods. The asymptotic expansions of Bahadur bounds and exponential rates in the case of the maximum likelihood estimator are obtained. Based on these results we have obtained a result parallel to the Fisher-Rao-Efron result concerning second-order efficiency (see Efron, 1975). Our results also substantiate the geometric observation given by Efron (1975) that if the statistical curvature of the underlying distribution is small, then the maximum likelihood estimator is nearly optimal.

Data collected from many independent identically distributed renewal processes, each of which is observed for an arbitrary period of time, is usually affected by censoring coupled with length biased sampling. In this paper we derive an algorithm that produces the nonparametric maximum likelihood estimator (i.e., the analog of the single-sample empirical distribution function) of the common lifetime distribution, based on such data.

A new methodology is proposed for discrimination among stationary time-series. The time series are transformed into binary arrays by clipping (retaining only the signs of) the $j$th difference series, $j = 0, 1, 2, \cdots$. The degeneracy of clipped $j$th differences is studied as $j$ becomes large. A new goodness of fit statistic is defined as a quadratic form in the counts of axis-crossings by each of the first $k$ differences of the series. Simulations and the degeneracy of high-order differences justify fixing $k$ no larger than 10 for many processes. Empirical simulated distributions (with $k = 9$) of the goodness of fit statistic suggest a gamma approximation for its tail probabilities. Illustrations are given of discrimination between simple models with the new statistic.

A class of probability density estimates can be obtained by penalizing the likelihood by a functional which depends on the roughness of the logarithm of the density. The limiting case of the estimates as the amount of smoothing increases has a natural form which makes the method attractive for data analysis and which provides a rationale for a particular choice of roughness penalty. The estimates are shown to be the solution of an unconstrained convex optimization problem, and mild natural conditions are given for them to exist. Rates of consistency in various norms and conditions for asymptotic normality and approximation by a Gaussian process are given, thus breaking new ground in the theory of maximum penalized likelihood density estimation.

We study the consistency properties of a nonparametric estimator $f_n$ of a density function $f$ on the real line, which is known as the "first MPLE of Good and Gaskins," and which is obtained by maximizing the likelihood functional multiplied by the roughness penality $\exp\{- \alpha \int (f'/f)^2 f\}$ with $\alpha > 0$. Under modest assumptions on the density function $f$, and letting $\alpha = \alpha_n \rightarrow \infty$ and $\alpha_n/n \rightarrow 0$ a.s. as $n \rightarrow \infty$ we demonstrate the a.s. convergence of $f_n$ to $f$, with rates, in the Hellinger, $L_1, L_2, \sup_{\mathbb{R}}$ and Sobolev norms, as well as in integrated mean absolute deviation. Finally, the corresponding estimator for $f$ supported on the half-line, is derived and the computational feasibility as well as the consistency properties of the estimator are indicated.

It is shown how the calculus can be used to characterize admissible decision rules (Pareto optimal points, efficient points). Necessary and sufficient conditions for admissibility are derived in terms of the first and the second directional derivatives of convex risk functions. In particular, the results obtained imply that if $p$ is to be estimated in the binomial distribution $B(n, p)$, then an estimator is admissible for the quadratic loss function if and only if it fulfills some analytic conditions.

Let $\mathbf{X} = (X_1, \cdots, X_p)^t$ be an observation from a $p$-variate normal distribution with unknown mean $\mathbf{\theta} = (\theta_1, \cdots, \theta_p)^t$ and identity covariance matrix. We consider a control problem which, in canonical form, is the problem of estimating $\mathbf{\theta}$ under the loss $L(\mathbf{\theta, \delta}) = (\mathbf{\theta}^t \mathbf{\delta} - 1)^2$, where $\mathbf{\delta(x)} = (\delta_1(\mathbf{x}), \cdots, \delta_p(\mathbf{x}))^t$ is the estimate of $\mathbf{\theta}$ for a given $\mathbf{x}$. General theorems are given for establishing admissibility or inadmissibility of estimators in this problem. As an application, it is shown that estimators of the form $\mathbf{\delta(x)} = (|\mathbf{x}|^2 + c)^{-1}\mathbf{x} + |\mathbf{x}|^{-4}w(|\mathbf{x}|)\mathbf{x}$, where $w(|\mathbf{x}|)$ tends to zero as $|\mathbf{x}| \rightarrow \infty$, are inadmissible if $c > 5 - p$, but are admissible if $c \leq 5 - p$ and $\mathbf{\delta}$ is generalized Bayes for an appropriate prior measure. Also, an approximation to generalized Bayes estimators for large $|\mathbf{x}|$ is developed.

Assume that $X_1, \cdots, X_p$ are independent random observations having discrete exponential densities $\rho_i(\theta_i)t_i(x_i)\theta^{xi}_i, i = 1, \cdots, p$ respectively. A general technique of improving upon the uniform minimum variance unbiased estimator (UMVUE) of $(\theta_1, \cdots, \theta_p)$ is developed under possibly weighted squared error loss functions. It is shown that improved estimators can be constructed by solving a difference inequality. Typical difference inequalities of a fairly general type are presented and solved. When specialized to Poisson and Negative binomial cases, broad classes of estimators are given that dominate the UMVUE. These results unify many known results in this rapidly diverging field, and some of them are new (especially those related to Negative Binomial distributions). Improved estimators are also obtained for the problems in which some of the observations are from Poisson families and some from Negative Binomial families. For sum of squared errors loss, estimators which dominate the UMVUE in the discrete exponential families are also given explicitly.

For the problem of estimating a $p$-variate normal mean, the existence of confidence procedures which dominate the usual one, a sphere centered at the observations, has long been known. However, no explicit procedure has yet been shown to dominate. For $p \geq 4$, we prove that if the usual confidence sphere is recentered at the positive-part James Stein estimator, then the resulting confidence set has uniformly higher coverage probability, and hence is a minimax confidence set. Moreover, the increase in coverage probability can be quite substantial. Numerical evidence is presented to support this claim.

We consider sequential tests with parabolic stopping boundary for the hypothesis that the drift of a Wiener process is positive. Formulas are derived for the power and expected sample size. The performance of minimax parabolic tests is compared with that of other tests considered in the literature.

Asymptotic expansions for the error probabilities of repeated significance tests about a normal mean are developed. Use of the expansions appears to result in substantially improved numerical accuracy, when compared to the use of the leading term, at least in some important special cases. The expansions are sufficiently refined to show the effect of some simple modifications of the basic procedure, such as requiring an initial sample size.

Let $X_1, X_2, \cdots$ be independent observations from a population with mean $\mu$ and variance $\sigma^2$, and suppose that given a sample of size $n$ one wishes to estimate $\mu$ by $\bar{X}_n$, subject to the loss function $L_n = A\sigma^{2\beta-2}(\bar{X}_n - \mu)^2 + n, A > 0, \beta > 0$. If $\sigma$ is known, then the optimal fixed sample size $n_0$ for minimizing the risk $R_n = EL_n$ can be computed, but if $\sigma$ is unknown there is no fixed sample size procedure that will achieve the minimum risk. For the case when $\sigma$ is unknown, a number of authors have investigated the performance of sequential estimation procedures designed to come close to attaining the minimum risk $R_{n_0}$. In this paper it is shown that for the class of sequential estimation procedures with stopping rules $T_A = \inf\{n \geq n_A: n^{-1} \sum^n_{i=1} (X_i - \bar{X}_n)^2 \leq A^{-1/\beta}n^{2/\beta}\}$ the regret $R_{T_A} - R_{n_0}$ remains bounded as $A\rightarrow \infty$, under suitable assumptions on the moments of $X_1$ and the delay $n_A$, but (unlike previous results of bounded regret) without any assumption about the type of distribution of $X_1$.

Some stationary and non-stationary time series arise from mixed distributions, the probabilities attached to the occurrence of certain values being positive, while a continuum of possible values is also involved. Such series are modeled in terms of a stationary Gaussian process $X_t$, which is censored when it crosses certain thresholds. Procedures are proposed for estimating the autocorrelation function of $X_t$. Their strong consistency and asymptotic normality are established. We suggest tests of the hypothesis that $X_t$ is white noise.

Theorems are proved relating to the rate of almost sure convergence of autocovariances, and hence autocorrelations, to their true values. These rates are uniform in the lag up to some order $P(T)$, increasing with $T$. The key assumption is that the process is stationary and the best linear predictor is the best predictor. In particular for an ARMA process and $P(T) = O\{(\ln T)^a\}, a < \infty$, the rate is $O\{(\ln \ln T/T)^{1/2}\}$. These results are used to discuss autoregressions and the use of autoregressions to approximate the structure of a more general process by increasing the order of the autoregression with $T$.

The idea of smoothing-spline interpolation is generalized to propose an estimator for the mean function of a stochastic process. A minimax property of the proposed estimator is then demonstrated under the usual squared loss function.

The general outlier problem for a multivariate normal random sample with mean slippage is defined and is shown to be invariant under a natural group of transformations. A family of maximal invariants is obtained, and the common distribution of its members is derived. The critical region for the locally best invariant test of the null hypothesis, that there are no outliers, versus the alternative hypothesis, that some outliers are present, is found. Under very general conditions, this test is equivalent to rejecting the null hypothesis whenever Mardia's multivariate sample kurtosis is sufficiently large.

This paper demonstrates the use of proper actions and quotient measures in representations of non-central distributions of maximal invariants.

In this paper we present minimal complete class theorems for testing problems with simple null hypotheses and multivariate one-sided alternative hypotheses. The results extend previous ones by not requiring that the tests be based on exponentially distributed random variables or that the null and alternative parameter spaces be topologically separated.

A statistical prediction problem invariant under a certain group of transformations is considered. Under several assumptions it is shown that the best invariant predictor is obtained by using the invariant measure on the group. This result is an extension of that of Hora and Buehler.

Let $f(x: \theta) = a(x) \exp\{\theta_1u_1(x) + \theta_2u_2(x) + c(\theta)\}, \theta = (\theta_1, \theta_2) \in \ominus \subset R^2$, be a density with respect to the Lebesgue measure on the real line which characterizes a two-parameter exponential family of distributions. Let $(\theta_1, \eta_2)$ be the mixed parameters, where $\eta_2 = E\{u_2(X)\}$. Assume that $\theta_2$ can be represented as $\theta_2 = -\theta_1 \varphi'(\eta_2)$ where $\varphi'(\eta_2) = d\varphi(\eta_2)/d\eta_2$ for some function $\varphi(\eta_2)$. Let $(X_1, \cdots, X_n)$ be independent random variables having a common density $f(x: \theta)$ and set $T_i = \sum^n_{j=1} u_i(X_j), i = 1, 2$. It is shown that if $u_2(x)$ is a 1-1 function then the random variables $T_2$ and $Z_n = T_1 - n\varphi(T_2/n)$ are independent and that the statistic $Z_n$ is ancillary for $\theta_2$ in the presence of $\theta_1$ (i.e. the density of $Z_n$ depends on $\theta_1$ only). Furthermore, the density of $Z_n$ belongs to the one-parameter exponential family with natural parameter $\theta_1$. These results enable us to construct uniformly most powerful unbiased (UMPU) tests for various hypotheses concerning the parameter $\theta_1$ which are based on the statistic $Z_n$.

A family of probability measures $\mathscr{P}$ on some measurable space $(X, \mathscr{A})$ and a class of estimator sequences $\hat{P}_n: X^n \rightarrow \mathscr{P}, n \in \mathbb{N}$, containing maximum likelihood estimators are considered. For $P \in \mathscr{P}$ it is proved that there are numbers $c > 0, h_0 > 0$ fulfilling $P^n\{n^{1/2} d(\hat{P}_n, P) > h\} \leq \exp(-ch^2)$ for $n \in \mathbb{N}, h \geq h_0$, where $d$ denotes the Hellinger distance of probability measures. Then parameterized families $\mathscr{P} = \{P(\theta): \theta \in \Theta\}$ are considered where $(\Theta, \Delta)$ is a separable and finite-dimensional metric space, and for sequences $\hat{\Theta}_n: X^n \rightarrow \Theta, n \in \mathscr{N}$, estimating the parameter similar inequalities are derived.

Simar (1976) suggested an iteration procedure for finding the maximum likelihood estimate of a compound Poisson process, but he could not show convergence. Here the more general case of maximizing a concave functional on the set of all probability measures is treated. As a generalization of Simar's procedure, an algorithm is given for solving this problem, including assumptions to ensure convergence to an optimum. Finally, it is shown that Simar's functional fulfills these assumptions.

The empirical intensity function of the Poisson point process is shown to be a nonparametric minimax estimator for a weighted quadratic loss function.

Let $X$ be distributed according to a non-central Chi squared distribution with $p$ degrees of freedom. The non-central Chi squared distribution arises in various statistical analyses and the estimation of the non-centrality parameter is of importance in some problems. This paper deals with the admissibility of certain estimates of the non-centrality parameter. It is shown that $(X - p)^+$, the positive part of $X - p$ dominates the maximum likelihood estimator with squared error as the loss function.

The use of admissible procedures in each of several problems may be inadmissible when the problems are combined, in the sense of summed loss. Thus apparently irrelevant information can sometimes be used to reduce risk over the entire parameter space. This is known as Stein's Paradox. We prove here that this cannot occur when the sample spaces for the problems are finite.

For a random vector $\mathbf{X}$ on $\mathbf{X} > \mathbf{b}$ whose mean exists, the mean residual lifetime $E(\mathbf{X} - \mathbf{c}\mid \mathbf{X} > \mathbf{c})$ is an affine function of $\mathbf{c}$ on $\mathbf{c} > \mathbf{b}$ if and only if $\mathbf{X}$ can be partitioned into independent random vectors which have shifted multivariate Pareto or exponential distributions. An interpretation in terms of income-distribution is suggested for the Pareto case. It is also shown that every multivariate distribution whose mean exists is determined by its mean residual lifetime.

In a Borel setting, every classically sufficient statistic is Bayes sufficient, but not vice versa. The example is a hypothesis testing problem in which Bayesians, but not classicists, can achieve zero error probabilities.

It is shown that several families of PBIB designs with relatively few blocks are $E$-optimal over the collection of all block designs. Among these are: the Partial Geometries with two associate classes; PBIB designs with $\lambda_1 = 1, \lambda_2 = 0$ and fewer blocks than varieties; PBIB designs with triangular schemes of size $n, \lambda_1 = 0, \lambda_2 = 1$ and block size $k \geq n/2$ (or $\lambda_1 = 1, \lambda_2 = 0$ and $k \geq n - 1)$; PBIB designs with $L_i$ schemes based on $\nu$ varieties with $\lambda_1 = 0, \lambda_2 = 1, k \geq \sqrt{\nu}$ (or $\lambda_1 = 1, \lambda_2 = 0$ and either $i - 1 \leq \sqrt{\nu} \leq k$ or $k \leq \sqrt{\nu} \leq i - 1$). The duals of these designs are also $E$-optimal. In certain settings Partial Geometries are the unique $E$-optimal designs.