We develop a unified framework within which many commonly used bootstrap critical points and confidence intervals may be discussed and compared. In all, seven different bootstrap methods are examined, each being usable in both parametric and nonparametric contexts. Emphasis is on the way in which the methods cope with first- and second-order departures from normality. Percentile-$t$ and accelerated bias-correction emerge as the most promising of existing techniques. Certain other methods are shown to lead to serious errors in coverage and position of critical point. An alternative approach, based on "shortest" bootstrap confidence intervals, is developed. We also make several more technical contributions. In particular, we confirm Efron's conjecture that accelerated bias-correction is second-order correct in a variety of multivariate circumstances, and give a simple interpretation of the acceleration constant.

Let $g$ be a nonlinear function of the regression parameters $\beta$ in a heteroscedastic linear model and $\hat{\beta}$ be the least squares estimator of $\beta.$ We consider the estimation of the variance and bias of $g(\hat{\beta})$ [as an estimator of $g(\beta)$] by using three resampling methods: the weighted jackknife, the unweighted jackknife and the bootstrap. The asymptotic orders of the mean squared errors and biases of the resampling variance and bias estimators are given in terms of an imbalance measure of the model. Consistency of the resampling estimators is also studied. The results indicate that the weighted jackknife variance and bias estimators are asymptotically unbiased and consistent and their mean squared errors are of order $o(n^{-2})$ if the imbalance measure converges to zero as the sample size $n \rightarrow \infty$. Furthermore, based on large sample properties, the weighted jackknife is better than the unweighted jackknife. The bootstrap method is shown to be asymptotically correct only under a homoscedastic error model. Bias reduction, a closely related problem, is also discussed.

Let $\Omega$ denote the parameter space of a statistical model and let $\mathscr{K}$ be the domain of variation of the parameter of interest. Various differential-geometric structures on $\Omega$ are considered, including the expected information metric and the $\alpha$-connections studied by Chentsov and Amari, as well as the observed information metric and the observed $\alpha$-connections introduced by Barndorff-Nielsen. Under certain conditions these geometric objects on $\Omega$ can be transferred in a canonical purely differential-geometric way to $\mathscr{K}$. The transferred objects are related to structures on $\mathscr{K}$ obtained from derivatives of pseudolikelihood functions such as the profile likelihood, the modified profile likelihood and the marginal likelihood based on an $L$-sufficient statistic (cf. Remon) when such a statistic exists. For composite transformation models it is shown that the modified profile likelihood is very close to the Laplace approximation to a certain integral representation of the marginal likelihood.

When there exist nuisance parameters whose number increases in proportion to that of independent observations, it is in general difficult to get a consistent or efficient estimator of a common structural parameter. The present paper proposes a new theory based on a vector bundle consisting of certain random variables over the statistical model. Structures and properties of estimating functions are elucidated in the class of consistent estimators. A necessary and sufficient condition is obtained for the existence of a consistent estimator given by an estimating function. A necessary and sufficient condition is then given for the existence of the optimal estimator in this class, which is further obtained when it exists. In their derivations, the concept of dual connections and parallel transports plays an essential role. The results are applied to a special type of exponential family, and the optimal estimators are explicitly obtained in some examples. This explains the reason why the conditional score plays an important role.

Vardi (1985a) introduced an $s$-sample model for biased sampling, gave conditions which guarantee the existence and uniqueness of the nonparametric maximum likelihood estimator $\mathbb{G}_n$ of the common underlying distribution $G$ and discussed numerical methods for calculating the estimator. Here we examine the large sample behavior of the NPMLE $\mathbb{G}_n$, including results on uniform consistency of $\mathbb{G}_n$, convergence of $\sqrt n (\mathbb{G}_n - G)$ to a Gaussian process and asymptotic efficiency of $\mathbb{G}_n$ as an estimator of $G$. The proofs are based upon recent results for empirical processes indexed by sets and functions and convexity arguments. We also give a careful proof of identifiability of the underlying distribution $G$ under connectedness of a certain graph $\mathbf{G}$. Examples and applications include length-biased sampling, stratified sampling, "enriched" stratified sampling, "choice-based" sampling in econometrics and "case-control" studies in biostatistics. A final section discusses design issues and further problems.

In this paper the asymptotic behaviour of the product limit estimator $F_n$ of an unknown distribution is investigated. We give an approximation of the difference $F_n(x) - F(x)$ by a Gaussian process and also by the average of i.i.d. processes. We get almost as good an approximation of the stochastic process $F_n(x) - F(x)$ as one can get for the analogous problem when the maximum likelihood estimator is approximated by a Gaussian random variable or by the average of i.i.d. random variables in the parametric case.

A general version of the Cox regression model is studied as a discrete time process consisting of the embedded experiments at the jump times. Simple conditions are given for the consistency and asymptotic normality of the estimator for the regression parameters.

In this paper, for right censored competing risks data, a class of tests developed for comparing the cumulative incidence of a particular type of failure among different groups. The tests are based on comparing weighted averages of the hazards of the subdistribution for the failure type of interest. Asymptotic results are derived by expressing the statistics in terms of counting processes and using martingale central limit theory. It is proposed that weight functions very similar to those for the $G^p$ tests from ordinary survival analysis be used. Simulation results indicate that the asymptotic distributions provide adequate approximations in moderate sized samples.

In a clinical trial to compare two treatments, suppose that patients are assigned to treatment according to a certain class of restricted randomization rules. Conditional on the difference in numbers between the two treatment groups, the asymptotic null distribution of a class of permutation tests is shown to be Gaussian.

We consider an arbitrary sequence of kernel density estimates $f_n$ with kernels $K_n$ possibly depending upon $n$. Under a mild restriction on the sequence $K_n$, we obtain inequalities of the type $E\big(\int|f_n - f|\big) \geq (1 + o(1))\Psi(n, f),$ where $f$ is the density being estimated and $\Psi(n, f)$ is a function of $n$ and $f$ only. The function $\psi$ can be considered as an indicator of the difficulty of estimating $f$ with any kernel estimate.

Let $(X_1, Y_1), \cdots, (X_n, Y_n)$ be a sample, denote the conditional density of $Y_i\mid X_i = x_i$ as $f(y\mid x_i, \theta(x_i))$ and $\theta$ an element of a metric space $(\Theta, d)$. A lower bound is provided for the $d$-error in estimating $\theta$. The order of the bound depends on the local behavior of the Kullback information of the conditional density. As an application, we consider the case where $\Theta$ is the space of $q$-smooth functions on $\lbrack 0, 1 \rbrack^d$ metrized with the $L_r$ distance, $1 \leq r < \infty$.

Run orders of fractional factorial designs which minimize a cost function based on the number of times the factors change levels during the time sequence in which the runs are performed and which simultaneously have all factor main effects components orthogonal to a polynomial time trend are found for a wide variety of factorial plans. A construction technique based on a generalized foldover scheme is presented.

Constructions and optimality results are given for block designs under first and second order (NN1 and NN2, respectively) neighbor correlations, extending the work of Kiefer and Wynn. Conditions for optimality and minimality are given for the NN2 model and new minimality results are found for the NN1 case. Construction of NN2 optimum complete block designs is solved and combinatorial arrays are used for NN2 optimum incomplete block designs. In many cases these are minimum optimum NN1 designs as well. A new solution for block size 3 is given. A method for constructing NN1 designs with partial variance balance is introduced and several series of these designs are shown to enjoy weaker optimality properties.

Polynomial regression in $q$ variables of degree $n$ on the $q$-cube is considered. Approximate $D$-optimal and approximate $D_s$-optimal designs for estimating higher degree terms are investigated. Numerical results are given for $n = 2$ with arbitrary $q$ and for $n = 3, 4, 5$ and $q = 2, 3$. Exact solutions are given within the class of product designs together with some efficiency calculations.

Robust fixed size confidence procedures are derived for the location parameter $\theta$ of a sample of $N$ i.i.d. observations of a scalar random variable $Z$ with CDF $F(z - \theta)$. Here, $\theta$ is restricted to a closed interval $\Omega$ and the uncertainty in $F$ is modeled by an uncertainty class $\mathscr{F}$. These robust confidence procedures are, in turn, based on the solution of a related robust minimax decision problem that is characterized by a zero-one loss function, the parameter space $\Omega$ and the uncertainty class $\mathscr{F}$. Sufficient conditions for the existence of robust minimax and robust median-minimax estimators are delineated. Sufficient conditions on $\mathscr{F}$ are obtained such that (i) both types of rules are minimax within the class of nonrandomized odd monotone procedures and (ii) subject to additional conditions, both types of rules are globally minimax admissible Bayes procedures. The paper concludes with an examination of the asymptotic behavior of the robust median-minimax estimators and their extensions to the robust $\alpha$-minimax rules, which are based on the $\alpha$-trimmed mean.

Given a random sample of size $n$ from an unknown continuous distribution function $F$, we consider the problem of estimating $F$ nonparametrically from a decision theoretic approach. In our treatment, we assume the Kolmogorov-Smirnov loss function and the group of all one-to-one monotone transformations of real numbers onto themselves which leave the sample values invariant. Under this setup, we obtain a best invariant estimator of $F$ which is shown to be unique. This estimator is a step function with unequal amounts of jumps at the observations and is an improper distribution function. It is remarked that this estimator may be used in constructing the best invariant confidence bands for $F$, and also in carrying out a goodness-of-fit test.

A class of proper scoring functions which combine the error in a decision problem and the precision of the statistical decision rule is introduced. The Bayesian procedures with respect to these loss functions are pairs formed by the usual Bayes decision and by the expected posterior loss. A necessary and sufficient condition for admissibility under the corresponding risk is given.

We study the empirical Bayes decision theory with an $m$-truncated sequential statistical decision problem as the component. An empirical Bayes sequential decision procedure is constructed for the linear loss two-action problem. Asymptotic results are presented regarding the convergence of the Bayes risk of the empirical Bayes sequential decision procedure. With sequential components, an empirical Bayes sequential decision procedure selects both a stopping rule function and a terminal decision rule function for use in the component with parameter $\theta$.

It is shown that in the stationary autoregressive case (Part I [Sections 1-5)] the distributions of least squares estimators and their close relatives such as sample autocovariances and the recently introduced general $M$-estimators converge to suitable Gaussian distributions uniformly over all Borel sets and uniformly over suitable neighborhoods of the parameter. Specifically, the notion of strongly asymptotically shift equivariance is introduced and it is shown that the distributions of any estimators satisfying this asymptotic equivariance condition converge in the preceding strong sense whenever they converge weakly (in law), provided the likelihood function of the sample is appropriately smooth. This smoothness of the likelihood is verified under mild conditions. Then, restricted to a broad class of models which include autoregressive models, a more easily verifiable condition implying the aforementioned asymptotic equivariance is derived and is shown to be satisfied by the estimators mentioned earlier. The methods used in the present paper are different from the usual method of characteristic functions; some indications of their possible wider scope are given. In Part II (Sections 6-8) the explosive autoregressive model is considered and a simple extension of the preceding result is applied to show that the least squares estimators converge in the preceding strong sense under suitable random or nonrandom normalization.

A procedure for estimating the parameters of a time series is proposed. The estimate minimizes a criterion function which is the weighted sum of squares of the distances between the periodogram and the spectrum of the series. Under regularity conditions, the estimate is shown to be strongly consistent. The asymptotic distribution of the estimate is also obtained. It is shown that, for a Gaussian process, an asymptotically efficient estimate can be obtained by using an iteratively reweighted procedure.

This paper presents necessary and sufficient conditions for the stochastic ordering of absolute univariate Gaussian random variables. This result is based on the behavior of the particular Gaussian density at zero, which is equivalent to restrictions on the mean and variance.

The problem is considered of estimating the point of global maximum of a function $f$ belonging to a class $F$ of functions on $\lbrack -1, 1 \rbrack,$ based on estimates of function values at points selected possibly during the experimentation. If $p$ is odd and greater than 1, $K$ is a positive constant and $F$ contains enough functions with $p$th derivatives bounded by $K$, then we prove that, under additional weak regularity conditions, the lower rate of convergence is $n^{-(p - 1)/(2p)}$.

Let $\{(X_i, \theta_i)\}$ be a sequence of independent random vectors where $X_i$ has a uniform density $U(0, \theta_i)$ for $0 < \theta_i < m (< \infty)$ and the unobservable $\theta_i$ are i.i.d. $G$ in some class $\mathscr{G}$ of prior distributions. In the $(n + 1)$st problem we estimate $\theta_{n + 1}$ by $t_n(X_1, \cdots, X_n, X_{n + 1}) \doteq t_n(\mathbf{X})$, incurring the risk $R_n \doteq \mathbf{E}(t_n(\mathbf{X}) - \theta_{n + 1})^2$, where $\mathbf{E}$ denotes expectation with respect to all random variables $\{(X_i, \theta_i)\}^{n + 1}_{i = 1}$. Let $R$ be the infimum Bayes risk with respect to $G$. In this paper the author exhibits empirical Bayes estimators with a convergence rate $O(n^{-1/2})$ of $R_n - R$ and shows that there is a sequence of empirical Bayes estimators for which $R_n - R$ has a lower bound of the same order $n^{-1/2}$.

In a parametric counterpart of the nonparametric considerations of Neuhaus, a class of quadratic goodness of fit statistics is developed generalizing the usual Cramer-von Mises statistics.