Let $X_1, \cdots, X_n$ be a random sample in $\mathbb{R}^m$ from an unknown density $f(x)$. Recently Bickel and Breiman have introduced a goodness of fit test for this situation based on the empirical distribution function of the variables $W_i = \exp\{-ng(X_i)V(R_i)\}, i = 1, \cdots, n$, where $g(x)$ is the hypothesized density and $V(R_i)$ represents the volume of the nearest neighbor sphere centered at $X_i$. Under the null hypothesis $H : f(x) = g(x)$, the empirical process is asymptotically independent of $g$. In this paper a weighted version of the empirical process is shown to produce tests which are still (essentially) distribution-free under $H$ but in addition have asymptotic power against sequences of alternatives contiguous to $g$. The optimal weight function is obtained as a function of the particular sequence of alternatives chosen, and consistency behavior against fixed alternatives is determined. Monte Carlo results illustrate the power performance of the tests for various densities and weight functions.

Let $X_1, \cdots, X_n$ be i.i.d. $\mathbb{R}^m$-valued observations from a bounded density $g(x)$ continuous on its support. Let $W_i = \exp\{- ng(X_i)V(R_i)\}, i = 1, \cdots, n$, where $V(R_i)$ is the volume of the nearest neighbor sphere around $X_i$, and let $w(x)$ be any bounded continuous weight function on $\mathbb{R}^m$. An infinite-dimensional approximation to the asymptotic form of the weighted empirical distribution function of the $W_i$'s is presented. The distributions of quadratic functionals of the limiting normalized weighted e.d.f. are found and tabulated for $m = \infty$ and $m = 1$ and compared with finite $m > 1$. Monte Carlo results are given for $n, m < \infty$.

Application of nonparametric density estimators generally requires the specification of a "smoothing parameter." The kernel estimator, for example, is not fully defined until a window width, or scaling, for the kernels has been chosen. Many "data-driven" techniques have been suggested for the practical choice of smoothing parameter. Of these, the most widely studied is the method of cross-validation. Our own simulations, as well as those of many other investigators, indicate that cross-validated smoothing can be an extremely effective practical solution. However, many of the most basic properties of cross-validated estimators are unknown. Indeed, recent results show that cross-validated estimators can fail even to be consistent for seemingly well-behaved problems. In this paper we will review the application of cross-validation to the smoothing problem, and establish $L_1$ consistency for certain cross-validated kernels and histograms.

In this paper it is shown that the Cox likelihood (Cox, 1972) may be treated as a standard likelihood, in the sense that its maximizer $\hat{\beta}$ is asymptotically normally distributed with asymptotic covariance matrix equal to $-\{E \partial^2 \log L (\beta)/\partial\beta\partial\beta'\}^{-1}$. In the process, an asymptotic representation of the score function is obtained in terms of functions of the independent observations. This representation may have some uses in itself such as: (1) providing a kind of residual for each observation, censored or uncensored, thereby indicating the relative influence of the observations, and (2) providing some information about the applicability of the asymptotics in a particular small sample. The asymptotic joint distribution of $\hat{\beta}$ and of the cumulative hazard function estimator $\hat{Lambda}_0(t)$ is also derived via a representation of the latter involving an independent increments process. Bailey (1982) shows that the "joint likelihood function" of the regression parameters $\beta$ and of the cumulative hazard jump parameters $\{\Lambda_i\}$ can be used in a natural way to obtain consistent estimates of these joint asymptotic covariances in the case of no ties. This justifies, to some extent, use of the general ML method for joint estimation of $\beta$ and $\Lambda_0(t)$.

Weak convergence results are proved for the product-limit estimator on the whole line. Applications are given to confidence band construction, estimation of mean lifetime, and to the theory of $q$-functions. The results are obtained using stochastic calculus and in probability linear bounds for empirical processes.

The connection between quasi-likelihood functions, exponential family models and nonlinear weighted least squares is examined. Consistency and asymptotic normality of the parameter estimates are discussed under second moment assumptions. The parameter estimates are shown to satisfy a property of asymptotic optimality similar in spirit to, but more general than, the corresponding optimal property of Gauss-Markov estimators.

A sequential sampling scheme in which observations are taken until the observed Fisher information exceeds a target value is considered for a restricted class of parametric families. It is proved that the theorems for fixed sample size asymptotic likelihood inference conditional on ancillary statistics are directly applicable to the sequential sampling scheme for location families. The fixed sample-size approximate ancillary statistic for two-dimensional curved exponential families is shown to be ancillary in the sequential sampling scheme. Evidence from Monte Carlo simulations suggests the applicability of the fixed sample size conditional likelihood inference theorems to these families as well. The distribution of the sequential sample size is shown to be asymptotically normal.

In this paper we consider the problem of estimating the parameters of a Markov model using so-called macro data. It will be shown that the stochastic process of the macro data is a Markov chain, which uniquely determines the probability structure of the underlying Markov model. A conditional least squares estimator exists under very weak conditions and this estimator is strongly consistent as time tends to infinity. Moreover this estimator is shown to be asymptotically normal under some additional assumptions.

In this paper certain fundamental properties of the maximum likelihood estimator of a mixing distribution are shown to be geometric properties of the likelihood set. The existence, support size, likelihood equations, and uniqueness of the estimator are revealed to be directly related to the properties of the convex hull of the likelihood set and the support hyperplanes of that hull. It is shown using geometric techniques that the estimator exists under quite general conditions, with a support size no larger than the number of distinct observations. Analysis of the convex dual of the likelihood set leads to a dual maximization problem. A convergent algorithm is described. The defining equations for the estimator are compared with the usual parametric likelihood equations for finite mixtures. Sufficient conditions for uniqueness are given. Part II will deal with a special theory for exponential family mixtures.

Two convergence aspects of the EM algorithm are studied: (i) does the EM algorithm find a local maximum or a stationary value of the (incomplete-data) likelihood function? (ii) does the sequence of parameter estimates generated by EM converge? Several convergence results are obtained under conditions that are applicable to many practical situations. Two useful special cases are: (a) if the unobserved complete-data specification can be described by a curved exponential family with compact parameter space, all the limit points of any EM sequence are stationary points of the likelihood function; (b) if the likelihood function is unimodal and a certain differentiability condition is satisfied, then any EM sequence converges to the unique maximum likelihood estimate. A list of key properties of the algorithm is included.

The pivotal model is described and applied to the estimation of parametric functions $\phi(\theta)$. This leads to equations of the form $H(x; \theta) = G\{p(x, \theta)\}$. These can be solved directly or by the use of differential equations. Examples include various parametric functions $\phi(\theta, \sigma)$ in a general location-scale distribution $f(p), p = (x - \theta)/\sigma$ and in two location-scale distributions. The latter case includes the ratio of the two scale parameters $\sigma_1/\sigma_2$, the difference and ratio of the two location parameters $\theta_1 - \theta_2$ and the common location $\theta$ when $\theta_1 = \theta_2 = \theta$. The use of the resulting pivotals to make inferences is discussed along with their relation to examples of non-uniqueness occurring in the literature.

If the sampling model is a continuous function of the parameter and if either the parameter space or the observation space is compact, then a coherent inference which is a continuous function of the observation must be the posterior of a proper, countably additive prior.

Genizi and Hochberg (1978) recommended using Contrast Set Preserving (CSP) procedures in the class of $T(Q)$ procedures for multiple comparisons in general unbalanced designs based on partial results. They did not, however, propose a general method for selecting a specific CSP procedure, or for replacing a given non-CSP procedure with a better CSP one. In this work we identify a certain orthogonal transformation of non-CSP procedures into CSP ones and give a sufficient condition for the uniform dominance (shorter confidence intervals for all contrasts) of the latter over the former. Two important implications of the given condition are: (i) Applying the given transformation to Spjotvoll and Stoline's (1973) $T$'-procedure in any unbalanced ANOVA gives a uniformly improved procedure. (ii) In any arbitrary design, our transformation gives uniform improvement if the original procedure is "nearly CSP."

Multi-stage hypothesis tests are studied as competitors of sequential tests. A class of three-stage tests for the one-dimensional exponential family is shown to be asymptotically efficient, whereas two-stage tests are not. Moreover, in order to be asymptotically optimal, three-stage tests must mimic the behavior of sequential tests. Similar results are obtained for the problem of testing two simple hypotheses.

The statistical properties of a cubic smoothing spline and its derivative are analyzed. It is shown that unless unnatural boundary conditions hold, the integrated squared bias is dominated by local effects near the boundary. Similar effects are shown to occur in the regularized solution of a translation-kernel integral equation. These results are derived by developing a Fourier representation for a smoothing spline.

In this paper we investigate an optimal property of maximum likelihood and quasi-maximum likelihood estimators of Gaussian autoregressive moving average processes by the second order approximation of the sampling distribution. It is shown that appropriate modifications of these estimators for Gaussian ARMA processes are second order asymptotically efficient if efficiency is measured by the degree of concentration of the sampling distribution up to second order. This concept of efficiency was introduced by Akahira and Takeuchi (1981).

Comparison of tests can be made in a local way using the concept of Pitman efficiency or in a non-local way using for instance Bahadur efficiency. In the local case the level of significance is kept fixed, whereas the non-local comparison typically deals with very small levels. These points of view may be seen as the extreme points of view. Here a possibly more realistic intermediate approach is introduced: as the number of observations tends to infinity the level is sent to zero, but not so very fast, in that way filling the gap between the Pitman and Bahadur approaches. The theory is exemplified by results on likelihood ratio tests, locally most powerful tests, and simple linear rank tests.

Wijsman (1959a) developed the theory of BAN estimators of a parameter $\beta$ under some fairly general conditions assuming that $n^{1/2}(y_n - g(\beta)) \rightarrow_L N_s(0, \sigma(\beta))$. The present article considers the complementary, but somewhat more general, approach under the constraint equation model that restricts the parameter $\mu$ so that $f(\mu) = 0$ under general conditions requiring $n^{1/2}(y_n - \mu) \rightarrow_L N_s(0, \sigma^\ast(\mu))$. At the same time, this article weakens Wijsman's differentiability requirement by introducing a $p$-differentiability condition for regular estimators. Next the theory of BAN estimation is developed for a model combining features of both of these approaches. As a special case of the model above, weighted least squares estimators for a general linear model are shown to be BAN.

In this note a generalization of a variance component test that was first suggested by Wald is examined. Necessary and sufficient conditions are given for the test to be applicable in a mixed linear model. A uniqueness property of the test in terms of degrees of freedom is also obtained.

In this paper we investigate the rates of convergence to zero of error probabilities in the estimation problem of a finite-valued parameter. It is shown that if a consistent estimator attains Bahadur's bound for the probability of incorrect decision at some parametric point then the error probability does not tend to zero exponentially fast for some other value of the parameter. We evaluate the minimal possible rate of convergence of this probability at a fixed parametric point for all asymptotically minimax procedures, and establish a necessary and sufficient condition for any of these procedures to have a constant risk. A simple example is constructed to demonstrate the asymptotical inadmissibility of the usual maximum likelihood estimator.

Probabilities in a large sparse contingency table are estimated by maximizing the likelihood modified by a roughness penalty. It is shown that if certain smoothness criteria on the underlying probability vector are met, the estimator proposed is consistent in a one-dimensional table under a sparse asymptotic framework. Suggestions are made for techniques to apply the estimator in practice, and generalization to higher dimensional tables is considered.

We consider the selection of the best subset of independent variables of a fixed size for possible inclusion in a regression model. The classical procedures (largest $\mathbb{R}^2$ to enter) are shown to be uniformly invariant Bayes in the sense of Paulson (1952) and Kudo (1956).

In many design settings where model violations are present, a "stochastic" minimaxity for many standard randomization procedures is demonstrated. This result requires no special analytic properties of the loss function and estimators. Next, under the squared loss and with the restriction to the use of linear estimators, a recipe for finding a randomized strategy is given. As a special case, randomizing an $A$-optimal design in the standard manner and using the least squares estimates yields a minimax strategy in most cases. These results generalize some aspects of Wu (1981).

Some methods for the construction of equineighbored balanced incomplete block designs introduced by Kiefer and Wynn (1981) are presented. An algorithm for constructing designs with $k = 3$ is developed. Kiefer and Wynn's result for $k = 3$ is difficult to implement in practice. Our algorithm provides a practical solution and makes use of the decomposition of complete graphs into disjoint Hamiltonian cycles. The construction of designs with $k = v - 1$ and $v - 2$ is also completely solved. The neighbor designs proposed for use in serology are useful for the construction of equineighbored balanced incomplete block designs. Several infinite families of equineighbored balanced incomplete block designs are listed.

The information matrices of one design in a finer and a simpler linear model are compared to each other. The orthogonality condition ensuring that both matrices are equal is examined in the model for repeated measurements designs which was considered e.g. by Cheng and Wu (1980). Examples of unbalanced designs fulfilling the orthogonality condition are shown to be optimum. Moreover, nearly strongly balanced generalized latin squares are introduced and their universal optimality is proved, if the numbers of units and periods are sufficiently large.

The problem of finding optimal incomplete blocks designs for comparing $p$ test treatments with a control is studied. B.I.B. designs are found to be $D$-optimal. $A$- and $E$-optimal designs are also obtained. For a large class of functions $\phi$, conditions for a design to be $\phi$-optimal are found. Most of the optimal designs are certain types of B.T.I.B. designs, introduced by Bechhofer and Tamhane (1981), which are binary in test treatments.

A life distribution $F$, with survival function $\bar{F} \equiv 1 - F$, is new better than used (NBU) if $\bar{F}(x + y) \leq \bar{F}(x)\bar{F}(y)$ for all $x, y \geq 0$. We propose a test of $H_0 : F$ is exponential, versus $H_1 : F$ is NBU, but not exponential, based on a randomly censored sample of size $n$ from $F$. Our test statistic is $J^c_n = \int \int \bar{F}_n(x + y) dF_n(x) dF_n(y)$, where $F_n$ is the Kaplan-Meier estimator. Under mild regularity on the amount of censoring, the asymptotic normality of $J^c_n$, suitably normalized, is established. Then using a consistent estimator of the null standard deviation of $n^{1/2}J^c_n$, an asymptotically exact test is obtained. We also study, using tests for the censored and uncensored models, the efficiency loss due to the presence of censoring.

Polynomial finite population functions can be expressed as totals over derived populations of batches, or ordered sequences of units from the original population. This paper extends the results of Godambe and Godambe and Joshi on nonexistence of best unbiased estimators and admissibility of the Horvitz-Thompson estimator to the real batch total case. The admissibility results are only partly extendible; an example is given to show that Horvitz-Thompson type estimators of the form $\sum \sum b_{ij}(y_i - y_j)^2/\pi_{ij}$ need not be admissible.

Random variables, $X_1, \cdots, X_k$ are said to be negatively associated (NA) if for every pair of disjoint subsets $A_1, A_2$ of $\{1, 2, \cdots, k\}, \operatorname{Cov}\lbrack f(X_1, i \in A_1), g(X_j, j \in A_2) \rbrack \leq 0$, for all nondecreasing functions $f, g$. The basic properties of negative association are derived. Especially useful is the property that nondecreasing functions of mutually exclusive subsets of NA random variables are NA. This property is shown not to hold for several other types of negative dependence recently proposed. One consequence is the inequality $P(X_i \leq x_i, i = 1, \cdots, k) \leq \prod^k_1P(X_i \leq x_i)$ for NA random variables $X_1, \cdots, X_k$, and the dual inequality resulting from reversing the inequalities inside the square brackets. In another application it is shown that negatively correlated normal random variables are NA. Other NA distributions are the (a) multinomial, (b) convolution of unlike multinomials, (c) multivariate hypergeometric, (d) Dirichlet, and (e) Dirichlet compound multinomial. Negative association is shown to arise in situations where the probability measure is permutation invariant. Applications of this are considered for sampling without replacement as well as for certain multiple ranking and selection procedures. In a somewhat striking example, NA and positive association representing quite strong opposing types of dependence, are shown to exist side by side in models of categorical data analysis.

Suppose that a statistician is faced with a decision problem involving an unknown parameter. Before making his decision he can carry out one of two possible experiments. Assume that he may choose at random which of the two experiments he will observe. For this problem a decision procedure for the statistician is a triple consisting of the randomizing probability measure he uses to choose between the experiments, the decision function he uses if he observes the first experiment and the decision function he uses if he observes the second experiment. The main theorem of this paper identifies the set of such admissible triples when the parameter space, and the sample spaces of the two experiments are finite. This result is then used to find some uniformly admissible procedures for some problems in finite population sampling.

The problem is to decide on the basis of repeated independent observations whether $\theta_0$ or $\theta_1$ is the true value of the parameter $\theta$ of a Koopman-Darmois family of densities, where the error probabilities are at most $\alpha_0$ and $\alpha_1$. An explicit method is derived for determining a combination of one-sided SPRT's, known, as a 2-SPRT, which minimizes the maximum expected sample size to within $o((\log \alpha^{-1}_0)^{1/2})$ as $\alpha_0$ and $\alpha_1$ go to 0, subject to the condition that $0 < C_1 < \log \alpha_0/\log\alpha_1 < C_2 < \infty$ for fixed but arbitrary constants $C_1$ and $C_2$. For the case of testing the mean of an exponential density, extensive computer calculations comparing the proposed 2-SPRT with optimal procedures show that the 2-SPRT comes within 2% of minimizing the maximum expected sample size over a broad range of error probability and parameter values.

Let $X_1, \cdots, X_n$ be independent with common distribution $F$ symmetric about $\mu$. Let $T_n = n^{-1} \sum^n_{i = 1} J(i/(n + 1))X_{in}$ be an $L$-estimate of $\mu$ based on a weight function $J$ and the order statistics $X_{1n} \leq \cdots \leq X_{nn}$ of $X_1, \cdots, X_n$. Under very general regularity conditions $n^{1/2}T_n$ has asymptotic variance $\sigma^2(J, F)$. A weight function $J_0$ is found that minimizes the maximum of $\sigma^2(J, F)/s^2(F)$, whenever $s(F)$ is a measure of scale of a general type, as $F$ ranges over a subclass of the symmetric distributions determined by $s(F)$ and $J$ ranges over a class of weight functions also determined by $s(F)$. The sample mean and the trimmed mean arise as the solutions for particular choices of scale measures.

The consistency of the least squares estimator of the parameter of the first order moving average time series is proven for the parameter in the interval $\lbrack -1, 1 \rbrack$.

It is shown that an equivariant statistic $S$ is invariantly sufficient iff the generated $\sigma$-algebra and the $\sigma$-algebra of the invariant Borel sets are independent, and that if $S$ is invariantly sufficient and equivariant, then the Pitman estimator for location parameter $\gamma$ is given by $S - E_0(S)$. For independent $X_1, \cdots, X_n$, the existence of an invariantly sufficient equivariant linear statistic is characterized by the normality of $X_1, \cdots, X_n$. Then, the independence of $X_1, \cdots, X_n$ is replaced by a linear framework in which there are established characterizations of the normality of $X = (X_1, \cdots, X_n)$ by properties (invariant sufficiency, admissibility, optimality) of the minimum variance unbiased linear estimator for $\gamma$.

It has been shown that the autocovariance matrices of a stationary multivariate time series can be uniquely characterized by a sequence of the normalized partial autocorrelation matrices having singular values less than one. In this note, we show that the same autocovariance matrices can be also uniquely characterized by a set of sequences of scalar partial autocorrelation coefficients whose magnitudes are all less than one.

Using a simple application of Fubini's theorem, we examine the connection between statistical independence, linear independence of random vectors, and algebraic independence of univariate r.v.'s, where we call a finite set of r.v.'s algebraically independent if they satisfy a non-trivial polynomial relationship only with zero probability. As a consequence, we simplify the derivation of a result of Eaton and Perlman (1973) on the linear independence of random vectors, and settle a matrix equation question of Okamoto (1973) concerning the rank of sample covariance-type matrices $S = XAX'$, where $X$ is $p \times n$, and $A$ is $n \times n$, for the case $n \geq p \geq r = \operatorname{rank}(A)$. We also derive a measure-theoretic version of the classical fact that the elementary symmetric polynomials in $m$ indeterminates are algebraically independent. This has applications to sample moments, $k$-statistics, and $U$-statistics with polynomial kernels.