The paper develops a one-step triangular array asymptotic expansion for continuous martingales which are asymptotically normal. Mixing conditions are not required, but the quadratic variations of the martingales must satisfy a law of large numbers and a central limit type condition. From this result we derive expansions for the distributions of estimators in asymptotically ergodic differential equation models, and also for the bootstrapping estimators of these distributions.

Bootstrap distributional limit theorems for $U$ and $V$ statistics are proved. They hold a.s., under weak moment conditions and without restrictions on the bootstrap sample size (as long as it tends to $\infty$), regardless of the degree of degeneracy of $U$ and $V$. A testing procedure based on these results is outlined.

The bootstrap is a poor estimator of bias in problems of curve estimation, and so bias must be corrected by other means when the bootstrap is used to construct confidence intervals for a probability density. Bias may either be estimated explicitly, or allowed for by undersmoothing the curve estimator. Which of these two approaches is to be preferred? In the present paper we address this question from the viewpoint of coverage accuracy, assuming a given number of derivatives of the unknown density. We conclude that the simpler, undersmoothing method is more efficacious. Undersmoothing also has advantages from the standpoint of minimizing interval width. We derive formulae for bandwidths which are optimal in terms of coverage accuracy and also give formulae for the coverage error which results from using those bandwidths.

Several authors have developed bootstrap methods for constructing confidence intervals in nonparametric regression. On each occasion a nonpivotal approach has been employed. Nonpivotal methods are still the overwhelmingly popular choice when statisticians use the bootstrap to compute confidence intervals, but they are not necessarily the most appropriate. In this paper we point out some of the theoretical advantages of pivoting. They include a reduction in the error of the bootstrap distribution function estimate, from $n^{-1/2}$ to $n^{-1}h^{-1/2}$ (where $h$ denotes bandwidth); and a reduction in coverage error of confidence intervals, from either $n^{-1/2}h^{-1/2}$ or $n^{-1/2}h^{1/2}$ (depending on which nonpivotal method is used) to $n^{-1}$. Several comparisons are drawn with the case of nonparametric density estimation, where a pivotal approach also reduces errors associated with confidence intervals, but where the orders of magnitude of the respective errors are quite different from their counterparts for nonparametric regression.

An exact and easily computable expression for the mean integrated squared error (MISE) for the kernel estimator of a general normal mixture density, is given for Gaussian kernels of arbitrary order. This provides a powerful new way of understanding density estimation which complements the usual tools of simulation and asymptotic analysis. The family of normal mixture densities is very flexible and the formulae derived allow simple exact analysis for a wide variety of density shapes. A number of applications of this method giving important new insights into kernel density estimation are presented. Among these is the discovery that the usual asymptotic approximations to the MISE can be quite inaccurate, especially when the underlying density contains substantial fine structure and also strong evidence that the practical importance of higher order kernels is surprisingly small for moderate sample sizes.

Estimators for location and size of a discontinuity or change-point in an otherwise smooth regression model are proposed. The assumptions needed are much weaker than those made in parametric models. The proposed estimators apply as well to the detection of discontinuities in derivatives and therefore to the detection of change-points of slope and of higher order curvature. The proposed estimators are based on a comparison of left and right one-sided kernel smoothers. Weak convergence of a stochastic process in local differences to a Gaussian process is established for properly scaled versions of estimators of the location of a change-point. The continuous mapping theorem can then be invoked to obtain asymptotic distributions and corresponding rates of convergence for change-point estimators. These rates are typically faster than $n^{-1/2}$. Rates of global $L^p$ convergence of curve estimates with appropriate kernel modifications adapting to estimated change-points are derived as a consequence. It is shown that these rates of convergence are the same as if the location of the change-point was known. The methods are illustrated by means of the well known data on the annual flow volume of the Nile river between 1871 and 1970.

It is argued that mean integrated squared error is not a useful measure of the performance of a variable bandwidth density estimator based on Abramson's square root law. The reason is that when the unknown density $f$ has even moderately light tails, properties of those tails drive the formula for optimal bandwidth, to the virtual exclusion of other properties of $f$. We suggest that weighted integrated squared error be employed as the performance criterion, using a weight function with compact support. It is shown that this criterion is driven by pointwise properties of $f$. Furthermore, weighted squared-error cross-validation selects a bandwidth which gives first-order asymptotic optimality of an adaptive, feasible version of Abramson's variable bandwidth estimator.

A Poisson approximation of a truncated, empirical point process enables us to reduce conditional statistical problems to unconditional ones. Let $(\mathbf{X,Y})$ be a $(d + m)$-dimensional random vector and denote by $F(\cdot\mid\mathbf{x})$ the conditional d.f. of $\mathbf{Y}$ given $\mathbf{X} = \mathbf{x}$. Applying our approach, one may study the fairly general problem of evaluating a functional parameter $T(F(\cdot\mid\mathbf{x}_1),\ldots,F(\cdot\mid\mathbf{x}_p))$ based on independent replicas $(\mathbf{X}_1,\mathbf{Y}_1),\ldots,(\mathbf{X}_n,\mathbf{Y}_n)$ of $(\mathbf{X,Y})$. This will be exemplified in the particular cases of nonparametric estimation of regression means and regression quantiles besides other functionals.

Suppose $X_1,X_2,\ldots,X_n$ are i.i.d. with unknown density $f$. There is a well-known expression for the asymptotic mean integrated squared error (MISE) in estimating $f$ by a kernel estimate $\hat{f}_n$, under certain conditions on $f$, the kernel and the bandwidth. Suppose that one would like to choose a sample size so that the MISE is smaller than some preassigned positive number $w$. Based on the asymptotic expression for the MISE, one can identify an appropriate sample size to solve this problem. However, the appropriate sample size depends on a functional of the density that typically is unknown. In this paper, a stopping rule is proposed for the purpose of bounding the MISE, and this rule is shown to be asymptotically efficient in a certain sense as $w$ approaches zero. These results are obtained for data-driven bandwidths that are asymptotically optimal as $n$ goes to infinity.

Suppose that the mean $\tau$ of a vector of Poisson variates is known to lie in a bounded domain $T$ in $\lbrack 0,\infty)^p$. How much does this a priori information increase precision of estimation of $\tau$? Using error measure $\sum_i(\hat\tau_i - \tau_i)^2/\tau_i$ and minimax risk $\rho(T)$, we give analytical and numerical results for small intervals when $p = 1$. Usually, however, approximations are needed. If $T$ is "rectangularly convex" at 0, there exist linear estimators with risk at most 1.26$\rho(T)$. For general $T, \rho(T) \geq p^2/(p + \lambda(\Omega))$, where $\lambda(\Omega)$ is the principal eigenvalue of the Laplace operator on the polydisc transform $\Omega = \Omega(T)$, a domain in twice-$p$-dimensional space. The bound is asymptotically sharp: $\rho(mT) = p - \lambda(\Omega)/m + o(m^{-1})$. Explicit forms are given for $T$ a simplex or a hyperrectangle. We explore the curious parallel of the results for $T$ with those for a Gaussian vector of double the dimension lying in $\Omega$.

In this paper we give necessary conditions and sufficient conditions for a test to be locally unbiased, we define local admissibility and we characterize local admissibility in hypothesis testing problems with simple null hypotheses. Applications are presented involving same-sign alternatives, ordered alternatives and independence testing of several variables.

We consider a biased sampling model that has been found useful in incorporating size biases inherent in many types of discovery data. The model postulates that the data are obtained from a finite population by selecting successively without replacement and with probability proportional to some measure of size. Unlike the ppswor scheme in survey sampling, it is assumed here that the size measure is a function of the unknown population values. In this article, we consider maximum likelihood estimation of the finite population parameters under this biased sampling model. We study the large sample behavior of the MLE's and derive a simple, asymptotically efficient approximation to the MLE. The approximate MLE is structurally similar to the Horvitz-Thompson estimator. We show that information about the order in the sample can be used to make inference even when the population size is unknown, which in fact can be estimated. Small sample behavior of the estimators is investigated through a limited simulation study, and the results are used to analyze oil and gas discovery data from the North Sea basin.

In the age-dependent minimal repair model of Block, Borges and Savits (BBS), a system failing at age $t$ undergoes one of two types of repair. With probability $p(t)$, a perfect repair is performed and the system is returned to the "good-as-new" state, while with probability $1 - p(t)$, a minimal repair is performed and the system is repaired, but is only as good as a working system of age $t$. Whitaker and Samaniego propose an estimator for the system life distribution $F$ when data are collected under this model. In the present article, an appropriate probability model for the BBS process is developed and a counting process approach is used to extend the large sample theorems of Whitaker and Samaniego to the whole line. Applications of these results to confidence bands and an extension of the Wilcoxon two-sample test are examined.

We consider a class of $U$-statistics type estimates for multivariate location. The estimates extend some $R$-estimates to multivariate data. In particular, the class of estimates includes the multivariate median considered by Gini and Galvani (1929) and Haldane (1948) and a multivariate extension of the well-known Hodges-Lehmann (1963) estimate. We explore large sample behavior of these estimates by deriving a Bahadur type representation for them. In the process of developing these asymptotic results, we observe some interesting phenomena that closely resemble the famous shrinkage phenomenon observed by Stein (1956) in high dimensions. Interestingly, the phenomena that we observe here occur even in dimension $d = 2$.

We introduce a simple procedure for obtaining robust estimates of the concentration parameter of the von Mises-Fisher distribution on the $q$-dimensional unit sphere. The procedure is based on the median deviation from a location parameter. Its influence function is derived and standardized bias robustness is proved. The asymptotic efficiency is calculated and an example is given.

A semiparametric method for estimating densities of normal mean mixtures is presented. This consistent data-driven method of estimation is based on probability spacings. The estimation technique involves iteratively fixing the standard deviation of the normal kernel that serves as a smoothing parameter, and then maximizing a function of the probability spacings over all mixing distributions. Based on the distribution of uniform spacings, a distribution free goodness-of-fit criterion is developed to guide the selection of the smoothing parameter. The result is a set of consistent estimators indexed by a range of smoothing parameters. Empirical process results are used to prove consistency.

Simple renormalization arguments can often be used to calculate optimal rates of convergence for estimating linear functionals from indirect measurements contaminated with white noise. This allows one to quickly identify optimal rates for certain problems of density estimation, nonparametric regression, signal recovery and tomography. Optimal kernels may also be derived from renormalization; we give examples for deconvolution and tomography.

A method of deriving asymptotics for linear processes is introduced which uses an explicit algebraic decomposition of the linear filter. The technique is closely related to Gordin's method but has some advantages over it, especially in terms of its range of application. The method offers a simple unified approach to strong laws, central limit theory and invariance principles for linear processes. Sample means and sample covariances are covered. The results accommodate both homogeneous and heterogeneous innovations as well as innovations with undefined means and variances.

This article studies the large sample behavior of the censored data least squares estimator derived from the synthetic data method proposed by Leurgans and Zheng. The asymptotic distributions are derived by representing the estimator as a martingale plus a higher-order remainder term. Recently developed counting process techniques are used. The results are then compared to the censored regression estimator of Koul, Susarla and Van Ryzin.

Consider an incomplete data problem with the following specifications. There are three independent samples $(X_1, \ldots, X_m), (Z_1, \ldots, Z_n)$ and $(U_1, \ldots, U_n)$. The first two samples are drawn from a common lifetime distribution function $G$, while the third sample is drawn from the uniform distribution over the interval $(0,1)$. In this paper we derive the large sample properties of $\hat{G}_{m,n}$, the nonparametric maximum likelihood estimate of $G$ based on the observed data $X_1, \ldots, X_m$ and $Y_1, \ldots, Y_n$, where $Y_i \equiv Z_iU_i, i = 1, \ldots, n$. (The $Z$'s and $U$'s are unobservable.) In particular we show that if $m$ and $n$ approach infinity at a suitable rate, then $\sup_t|\hat{G}_{m,n}(t) - G(t)| \rightarrow 0$ (a.s.), $\sqrt{m + n}(\hat{G}_{m,n} - G)$ converges weakly to a Gaussian process and the estimate $\hat{G}_{m,n}$ is asymptotically efficient in a nonparametric sense.

Sliced inverse regression [Li (1989), (1991) and Duan and Li (1991)] is a nonparametric method for achieving dimension reduction in regression problems. It is widely applicable, extremely easy to implement on a computer and requires no nonparametric smoothing devices such as kernel regression. If $Y$ is the response and $X \in \mathbf{R}^p$ is the predictor, in order to implement sliced inverse regression, one requires an estimate of $\Lambda = E\{\operatorname{cov}(X\mid Y)\} = \operatorname{cov}(X) - \operatorname{cov}\{E(X\mid Y)\}$. The inverse regression of $X$ on $Y$ is clearly seen in $\Lambda$. One such estimate is Li's (1991) two-slice estimate, defined as follows: The data are sorted on $Y$, then grouped into sets of size 2, the covariance of $X$ is estimated within each group and these estimates are averaged. In this paper, we consider the asymptotic properties of the two-slice method, obtaining simple conditions for $n^{1/2}$-convergence and asymptotic normality. A key step in the proof of asymptotic normality is a central limit theorem for sums of conditionally independent random variables. We also study the asymptotic distribution of Greenwood's statistics in nonuniform cases.

For random vectors taking values in $\mathbb{R}^d$ we introduce a notion of multivariate quantiles defined in terms of a class of sets and study an associated process which we call the generalized quantile process. This process specializes to the well known univariate quantile process. We obtain functional central limit theorems for our generalized quantile process and show that both Gaussian and non-Gaussian limiting processes can arise. A number of interesting example are included.

Matched sampling is a common technique used for controlling bias in observational studies. We present a general theoretical framework for studying the performance of such matching methods. Specifically, results are obtained concerning the performance of affinely invariant matching methods with ellipsoidal distributions, which extend previous results on equal percent bias reducing methods. Additional extensions cover conditionally affinely invariant matching methods for covariates with conditionally ellipsoidal distributions. These results decompose the effects of matching into one subspace containing the best linear discriminant, and the subspace of variables uncorrelated with the discriminant. This characterization of the effects of matching provides a theoretical foundation for understanding the performance of specific methods such as matched sampling using estimated propensity scores. Calculations for such methods are given in subsequent articles.

A basic interpretation is given which provides a new way of understanding the structure of the species problem and which leads to the popular Turing-Good-Robbins estimator. Through this interpretation we provide an explanation why the Turing-Good-Robbins estimators are always biased. An iterative procedure is suggested and applied to these estimators, which leads to new estimators whose biases are reduced. Using this basic construction we are able to generalize our discussion to a much broader class of coverage problems with the species problem as a special case. Three examples are studied in detail: the species problem, the problem of estimating the volume of a convex set and the missile-coverage problem. Furthermore, we derive general (new) estimators and study their properties by applying the interpretation to the framework of the general coverage problem. It is pointed out that, as in species problem, the general estimators derived from the interpretation are usually biased, we then apply our construction together with the iterative procedure to the previous three examples to produce new estimators whose biases are reduced. Finally, we extend our construction to the conditional cases.

Suppose the variable $X$ to be predicted and the learning sample $Y_n$ that was observed are independent, with a joint distribution that depends on an unknown parameter $\theta$. A prediction region $D_n$ for $X$ is a random set, depending on $Y_n$, that contains $X$ with prescribed probability $\alpha$. In sufficiently regular models, $D_n$ can be constructed so that overall coverage probability converges to $\alpha$ at rate $n^{-r}$, where $r$ is any positive integer. This paper shows that the conditional coverage probability of $D_n$, given $Y_n$, converges in probability to $\alpha$ at a rate which usually cannot exceed $n^{-1/2}$.

The problem of maximin testing between families of probability measures generated by special capacities and transformation groups is solved by showing that the Hunt-Stein theorem is applicable to such, generally nondominated, families of probability measures and that the problem reduces to maximin testing between neighbourhoods of distributions of maximal invariant, defined by special capacities closely related to the initial ones. An application to testing approximate binormality is also sketched.

Several authors have shown how to efficiently estimate $\beta$ in the semiparametric additive model $y = x'\beta + g(t) + \text{error}$, $g(t)$ smooth but unknown when the error distribution is normal. However, the general theory suggests that efficient estimation should be possible for general error distributions with finite Fisher information even when the error distribution is unknown. In this note we construct a sequence of estimators which achieves this goal under technical assumptions.

Vincentization is a convenient method of aggregating a set of $n \geq 2$ probability distributions $F_1, \ldots, F_n$ in such a way that their synthesis, $F = T(F_1, \ldots, F_n)$, be of the same functional form as the $F_i$'s when the latter are identical up to a location-scale transformation. A characterization of this combination rule is proposed and some of its consequences are outlined.

In this paper, we prove the uniqueness of linear i.i.d. representations of non-Gaussian linear processes on a countable abelian group under a basic invertibility condition, without requiring the existence of higher than second moments.