We discuss the following problem: given a random sample $\mathbf{X} = (X_1, X_2, \cdots, X_n)$ from an unknown probability distribution $F$, estimate the sampling distribution of some prespecified random variable $R(\mathbf{X}, F)$, on the basis of the observed data $\mathbf{x}$. (Standard jackknife theory gives an approximate mean and variance in the case $R(\mathbf{X}, F) = \theta(\hat{F}) - \theta(F), \theta$ some parameter of interest.) A general method, called the "bootstrap," is introduced, and shown to work satisfactorily on a variety of estimation problems. The jackknife is shown to be a linear approximation method for the bootstrap. The exposition proceeds by a series of examples: variance of the sample median, error rates in a linear discriminant analysis, ratio estimation, estimating regression parameters, etc.

The framework for multistage comparison procedures in the present paper is roughly that introduced by Duncan and treated more fully by Tukey. In the present paper we consider the problem of finding the optimum allocation of nominal significance levels for successive stages. The optimum procedure we obtain when the number $s$ of treatments is odd, and the compromise procedure we propose for even $s$, essentially agree with a procedure suggested by Tukey. The agreement is exact when $s$ is even and close when $s$ is odd. The results of the present paper apply among others to the problem of distinguishing normal distributions with known variances, multinomial distributions, Poisson distributions, and distributions in certain nonparametric settings. However, they do not apply exactly to the comparison of normal distributions with a common unknown variance. When the variances are completely unknown, the method applies in principle but faces the difficulty that no exact test is then available for testing the homogeneity of a set of means.

While usual sequential analysis deals with i.i.d. observations, this paper studies sequential tests for the dependent case of sampling without replacement from a finite population. A general weak convergence theorem is obtained and it is applied to the asymptotic analysis of the tests. Motivated by such applications as election predictions and acceptance sampling, the case of hypergeometric populations is studied in detail and a simple test with a triangular continuation region is proposed and is shown to have many nice properties. The paper concludes with a general heuristic principle of "finite-population correction" which is applicable to both sequential testing and fixed-width interval estimation problems.

This paper continues earlier work of the authors. An analogue of Blackwell's renewal theorem is obtained for processes $Z_n = S_n + \xi_n$, where $S_n$ is the $n$th partial sum of a sequence $X_1, X_2, \cdots$ of independent identically distributed random variables with finite positive mean and $\xi_n$ is independent of $X_{n+1}, X_{n+2}, \cdots$ and has sample paths which are slowly changing in a sense made precise below. As a consequence, asymptotic expansions up to terms tending to 0 are obtained for the expected value of certain first passage times. Applications to sequential analysis are given.

A new asymptotic theory for studying the effect of dependence of the observations in experimental design for the linear model is developed. The uniform design is shown to be asymptotically optimal in a strong sense for estimating location and in a weaker sense for estimating the slope of a straight line regression. Numerical results supporting the asymptotics appear in a companion paper.

The relation between fitting autoregression and periodogram is herein presented. More specifically, the asymptotic error covariance matrices of the estimates of the autoregressive parameters using Yule-Walker equations are expressed in terms of the periodogram. These expressions permit the immediate calculation of the error covariance matrices for various time series problems including the autoregressive spectral estimation and fitting autoregression to the data with randomly missed observations.

It is well known that the limit distribution of the supremum of the empirical distribution function $F_n$ centered at its expectation $F$ and standardized by division by its standard deviation is degenerate, if the supremum is taken on too large regions $\varepsilon_n < F(u) < \delta_n$. So it is natural to look for sequences of linear transformations, so that for given sequences of sup-regions $(\varepsilon_n, \delta_n)$ the limit of the transformed sup-statistics is nondegenerate. In this paper a partial answer is given to this problem, including the case $\varepsilon_n \equiv 0, \delta_n \equiv 1$. The results are also valid for the Studentized version of the above statistic, and the corresponding two-sided statistics are treated, too.

The supremum of the empirical distribution function $F_n$ centered at its expectation $F$ and standardized by division by its standard deviation has recently been shown by Jaeschke to have asymptotically an extreme-value distribution after a second location and scale transformation depending only on the sample size $n$. In this paper the studentized form of the above statistic, obtained by division by the estimated standard deviation, is shown to have the same large sample behavior. This statement is equivalent to the analogous assertion for the standardized sample quantile process for the uniform distribution. The three results imply each other. The present result yields immediately confidence regions that contract to zero width in the tails. The proofs given here rest on a limit theorem by Darling and Erdos on the maxima of standardized partial sums of i.i.d. random variables. In addition, Kolmogorov's theorem is used.

The objective in nonparametric regression is to infer a function $m(x)$ on the basis of a finite collection of noisy pairs $\{(X_i, m(X_i) + N_i)\}^n_{i=1}$, where the noise components $N_i$ satisfy certain lenient assumptions and the domain points $X_i$ are selected at random. It is known a priori only that $m$ is a member of a nonparametric class of functions (that is, a class of functions like $C\lbrack 0, 1\rbrack$ which, under customary topologies, does not admit a homeomorphic indexing by a subset of a Euclidean space). The main theoretical contribution of this study is to derive uniform convergence bounds and uniform consistency on bounded intervals for the Nadaraya-Watson kernel estimator and its derivatives. Also, we obtain the corresponding convergence results for the Priestly-Chao estimator in the case that the domain points are nonrandom. With these developments we are able to apply nonparametric regression methodology to the problem of identifying noisy time-varying linear systems.

Let $Z$ be a random vector whose distribution is spherically symmetric about the origin. A random vector $X$ which is representable as the image of $Z$ under affine transformation is said to have an ellipsoidally symmetric distribution. The model of ellipsoidal symmetry is a useful generalization of multivariate normality. This paper proposes and studies some goodness-of-fit tests which have good asymptotic power over a broad spectrum of alternatives to ellipsoidal symmetry.

Let $X_1, \cdots, X_n$ be a random sample from an unknown $\operatorname{cdf} F$, let $y_1, \cdots, y_n$ be known real constants, and let $Z_i = \min(X_i, y_i), i = 1, \cdots, n$. It is required to estimate $F$ on the basis of the observations $Z_1, \cdots, Z_n$, when the loss is squared error. We find a Bayes estimate of $F$ when the prior distribution of $F$ is a process neutral to the right. This generalizes results of Susarla and Van Ryzin who use a Dirichlet process prior. Two types of censoring are introduced--the inclusive and exclusive types--and the class of maximum likelihood estimates which thus generalize the product limit estimate of Kaplan and Meier is exhibited. The modal estimate of $F$ for a Dirichlet process prior is found and related to work of Ramsey. In closing, an example illustrating the techniques is given.

For exponential families with density \begin{equation*}x \rightarrow C(\theta, \eta)h(x)\exp\lbrack a(\theta)T(x) + \Sigma^p_{i=1} a_i(\theta, \eta)S_i(x)\rbrack, (\theta, \eta) \in \Theta \times H, \Theta \subset \mathbb{R},\end{equation*} $a$ increasing and continuous, there exists for every sample size an estimator for $\theta$ which is--in the class of all median unbiased estimators--of minimal risk for any monotone loss function.

In the general linear model $\mathscr{E}(\mathbf{y}) = \mathbf{X\beta}$, the vector $\mathbf{A\beta}$ is estimable whenever there is a matrix $\mathbf{B}$ so that $\mathscr{E}(\mathbf{By}) = \mathbf{A\beta}$. Several characterizations of estimability are presented along with short easy proofs. The characterizations involve rank equalities, generalized inverses, Schur complements and partitioned matrices.

Let $X_i$ and $Y_i$ be random variables related to other random variables $U_i, V_i$, and $W_i$ as follows: $X_i = U_i + W_i, Y_i = \alpha + \beta U_i + V_i, i = 1, \cdots, n$, where $\alpha$ and $\beta$ are finite constants. Here $X_i$ and $Y_i$ are observable while $U_i, V_i$ and $W_i$ are not. This model is customarily referred to as the regression problem with errors in both variables and the central question is the estimation of $\beta$. We give a class of estimates for $\beta$ which are asymptotically normal with mean $\beta$ and variance proportional to $1/n^{\frac{1}{2}}$, under weak assumptions. We then show how to choose a good estimate of $\beta$ from this class.

Let $X_1, \cdots, X_k$ be independent random variables with distributions $Q_1, \cdots, Q_k$ defined on a common range space $(\mathscr{X}, \mathscr{G})$. At the beginning it is assumed that the $Q_i$ are known but not the pairing with the $X_i$, and the goal is to identify the $X_i$ which comes from $Q_1$. First it is shown that every procedure based on a total ordering in $\mathscr{X}$ can be viewed as being based on (the $p$-value of) a test for deciding between $H_0: \{Q_1\}$ versus $H_1: \{Q_2, \cdots, Q_k\}$. Then the class of procedures based on tests is studied in detail. It is demonstrated how typical properties of tests $\varphi$ (powerfulness, unbiasedness, consistency etc.) transfer to the corresponding procedures $S_\varphi$. The next step is to get free of the assumption that $Q_1, \cdots, Q_k$ are known, thereby passing over from identification to selection procedures. Throughout this paper the objective is to compare procedures (not to establish specified ones), and as one main result it is shown that the asymptotic relative efficiency (Pitman) of one test $\varphi$ with respect to a second test $\psi$ and of $S_\varphi$ with respect to $S_\psi$ are identical.

David summarized distribution-free bounds for $E(X_{k:n})$, the expected value of the $k$th order statistic, and for the expected value of certain linear combinations of the order statistics, when sampling $n$ i.i.d. observations from a population with expectation $\mu$ and variance $\sigma^2$. Here the problem of finding distribution-free bounds for the expectations of linear systematic statistics is considered in the case in which the observations $X_i, i = 1,2, \cdots, n$, satisfy only $E(X_i) = \mu$ and $\operatorname{Var}(X_i) = \sigma^2$. The observations may be dependent and have different distributions. Bounds are obtained for the expectations of the $k$th order statistic, the trimmed mean, the range, and quasi-ranges, the spacings and Downton's estimator of $\sigma$. The sharpness of these bounds is considered. In contrast with the i.i.d. case all the bounds obtained are shown to be sharp.

Results of an earlier paper giving first order optimal robust $M$-estimators of a location parameter in certain dependent situations are extended to the case where the dependency can be modeled by a symmetric form of a $(2k + 1)$st order moving average scheme. It is also shown that the results can not be extended to finding second order optimal $M$-estimators. That is, no $M$-estimators can be optimal to the second order unless they explicitly adapt themselves to the assumed model for dependency.

We study general multiparameter exponential families of distribution and obtain differential equations relating the first two moments of the sufficient statistics to the normalization constant. Another result illuminates the structure of both the second order partial derivatives of the likelihood and their expected values.