Consider a general regression model of the form $y = g(\alpha + \mathbf{x}'\beta, \varepsilon)$, with an arbitrary and unknown link function $g$. We study a link-free method, the slicing regression, for estimating the direction of $\beta$. The method is easy to implement and does not require iterative computation. First, we estimate the inverse regression function $E(\mathbf{x}\mid y)$ using a step function. We then estimate $\Gamma = \operatorname{Cov}\lbrack E(\mathbf{x}\mid y)\rbrack$, using the estimated inverse regression function. Finally, we take the spectral decomposition of the estimate $\hat\Gamma$ with respect to the sample covariance matrix for $\mathbf{x}$. The principal eigenvector is the slicing regression estimate for the direction of $\beta$. We establish $\sqrt n$-consistency and asymptotic normality, derive the asymptotic covariance matrix and provide Wald's test and a confidence region procedure. Efficiency is discussed for an important special case. Most of our results require $\mathbf{x}$ to have an elliptically symmetric distribution. When the elliptical symmetry is violated, a bias bound is provided; the asymptotic bias is small when the elliptical symmetry is nearly satisfied. The bound suggests a projection index which can be used to measure the deviation from elliptical symmetry. The theory is illustrated with a simulation study.

A class of rank estimators is introduced for regression analysis in the presence of both left-truncation and right-censoring on the response variable. By making use of martingale theory and a tightness lemma for stochastic integrals of multiparameter empirical processes, the asymptotic normality of the estimators is established under certain assumptions. Adaptive choice of the score functions to give asymptotically efficient rank estimators is also discussed.

We study the asymptotics of the ML estimators for the Curie-Weiss model parametrized by the inverse temperature $\beta$ and the external field $h$. We show that if both $\beta$ and $h$ are unknown, the ML estimator of $(\beta, h)$ does not exist. For $\beta$ known, the ML estimator $\hat{h}_n$ of $h$ exhibits, at a first order phase transition point, superefficiency in the sense that its asymptotic variance is half of that of nearby points. At the critical point $(\beta = 1)$, if the true value is $h = 0$, then $n^{3/4}\hat h_n$ has a non-Gaussian limiting law. Away from phase transition points, $\hat h_n$ is asymptotically normal and efficient. We also study the asymptotics of the ML estimator of $\beta$ for known $h$.

This paper discusses the minimum Hellinger distance estimation (MHDE) of the parameter that gives the "best fit" of a parametric family to a density when the data are randomly censored. In studying the MHDE, the tail behavior of the product-limit (P-L) process is investigated, and the weak convergence of the process on the real line is established. An upper bound on the mean square increment of the normalized P-L process is also obtained. With these results, the asymptotic behavior of the MHDE is established and it is shown that, when the parametric model is correct, the MHD estimators are asymptotically efficient among the class of regular estimators. This estimation procedure is also minimax robust in small Hellinger neighborhoods of the given parametric family. The work extends the results of Beran for the complete i.i.d. data case to the censored data case. Some of the proofs employ the martingale techniques by Gill.

An approximate maximum likelihood estimate is known to be consistent under some compactness and integrability conditions. In this paper we study its convergence rate and its asymptotic efficiency in estimating smooth functionals of the parameter. We provide conditions under which the rate of convergence can be established. This rate is essentially governed by the size of the space of score functions as measured by an entropy index. We also show that, for a large class of smooth functionals, the plug-in maximum likelihood estimate is asymptotically efficient, that is, it achieves the minimal Fisher information bound. The theory is illustrated by several nonparametric or semiparametric examples.

Consider estimating a functional $T(F)$ of an unknown distribution $F \in \mathbf{F}$ from data $X_1, \cdots, X_n$ i.i.d. $F$. Let $\omega(\varepsilon)$ denote the modulus of continuity of the functional $T$ over $\mathbf{F}$, computed with respect to Hellinger distance. For well-behaved loss functions $l(t)$, we show that $\inf_{T_n \sup_\mathbf{F}} E_Fl(T_n - T(F))$ is equivalent to $l(\omega(n^{-1/2}))$ to within constants, whenever $T$ is linear and $\mathbf{F}$ is convex. The same conclusion holds in three nonlinear cases: estimating the rate of decay of a density, estimating the mode and robust nonparametric regression. We study the difficulty of testing between the composite, infinite dimensional hypotheses $H_0: T(F) \leq t$ and $H_1: T(F) \geq t + \Delta$. Our results hold, in the cases studied, because the difficulty of the full infinite-dimensional composite testing problem is comparable to the difficulty of the hardest simple two-point testing subproblem.

We establish upper and lower bounds on the asymptotic minimax risk in estimating (1) a density at a point when the density is known to be decreasing with a Lipschitz condition; (2) a density at a point when the density satisfies a local second-order smoothness (Sacks-Ylvisaker) condition; and (3) the $k$th derivative of the density at a point, when the density satisfies a local $L_p$ constraint on the $m$th derivative. In (1), (2) and (3) the upper and lower bounds differ asymptotically by less than 18%, 24.3% and 25%, respectively. Our bounds on the asymptotic minimax risk come from a simple formula. Let $\omega(\varepsilon)$ denote the modulus of continuity, with respect to Hellinger distance, of the functional to be estimated; in the previous cases this has the form $\omega(\varepsilon) = A\varepsilon^r(1 + o(1))$ for certain constants $A$ and $r$. Then, in all these cases, the minimax risk is not larger asymptotically than $r^r(1 - r)^{1 - r}\omega^2(n^{-1/2})/4$ and is at best a few percent smaller. The modulus of continuity of the functional and hence the geometry of the problem, determine the difficulty of estimation. At a technical level, two interesting aspects of our work are (1) derivation of minimax affine estimates of a linear functional in the white noise model with general convex asymmetric a priori class and (2) the use of Le Cam's theory of convergence of experiments to show that the density model is asymptotically just as hard as the white noise model. At a conceptual level, an interesting aspect of our work is the use of the hardest one-dimensional subproblem heuristic. Our method works because in these cases, the difficulty of the hardest one-dimensional subproblem is essentially equal to the difficulty of the full infinite-dimensional problem.

We consider here spline estimates of the optimal transformations of variables for multiple correlation and regression as dealt with in a recent paper by Breiman and Friedman. We show that we can construct estimates of the optimal transformations which have the same optimal rate of convergence as in the usual nonparametric estimation of a univariate function.

The problem of estimating a smooth monotone regression function $m$ will be studied. We will consider the estimator $m_{SI}$ consisting of a smoothing step (application of a kernel estimator based on a kernel $K$) and of a isotonisation step (application of the pool adjacent violator algorithm). The estimator $m_{SI}$ will be compared with the estimator $m_{IS}$ where these two steps are interchanged. A higher order stochastic expansion of these estimators will be given which show that $m_{SI}$ and $m_{SI}$ are asymptotically first order equivalent and that $m_{IS}$ has a smaller mean squared error than $m_{SI}$ if and only if the kernel function of the kernel estimator is not too smooth.

We propose a new nonparametric regression estimate. In contrast to the traditional approach of considering regression functions whose $m$th derivatives lie in a ball in the $L_\infty$ or $L_2$ norm, we consider the class of functions whose $(m - 1)$st derivative consists of at most $k$ monotone pieces. For many applications this class seems more natural than the classical ones. The least squares estimator of this class is studied. It is shown that the speed of convergence is as fast as in the classical case.

Let $(X, Y)$ be a random vector such that $X$ is $d$-dimensional, $Y$ is real valued and $Y = \theta(X) + \varepsilon$, where $X$ and $\varepsilon$ are independent and the $\alpha$th quantile of $\varepsilon$ is $0$ ($\alpha$ is fixed such that $0 < \alpha < 1$). Assume that $\theta$ is a smooth function with order of smoothness $p > 0$, and set $r = (p - m)/(2p + d)$, where $m$ is a nonnegative integer smaller than $p$. Let $T(\theta)$ denote a derivative of $\theta$ of order $m$. It is proved that there exists a pointwise estimate $\hat{T}_n$ of $T(\theta)$, based on a set of i.i.d. observations $(X_1, Y_1),\cdots,(S_n, Y_n)$, that achieves the optimal nonparametric rate of convergence $n^{-r}$ under appropriate regularity conditions. Further, a local Bahadur type representation is shown to hold for the estimate $\hat{T}_n$ and this is used to obtain some useful asymptotic results.

Simultaneous error bars are constructed for nonparametric kernel estimates of regression functions. The method is based on the bootstrap, where resampling is done from a suitably estimated residual distribution. The error bars are seen to give asymptotically correct coverage probabilities uniformly over any number of gridpoints. Applications to an economic problem are given and comparison to both pointwise and Bonferroni-type bars is presented through a simulation study.

Biased sampling regression models were introduced by Jewell, generalizing the truncated regression model studied by Bhattacharya, Chernoff and Yang. If the independent variable takes on only a finite number of values (as does the stratum variable), we show: 1. That if the slope of the underlying regression model is assumed known, then the nonparametric maximum likelihood estimates of the distribution of the independent and dependent variables (a) can be calculated from ordinary $M$ estimates; (b) are asymptotically efficient. 2. How to construct $M$ estimates of the slope which are always $\sqrt n$ consistent, asymptotically Gaussian and are efficient locally, for example, if the error distribution is Gaussian. We support our asymptotics with a small simulation.

It has been observed that to a smoothing spline operator there corresponds an equivalent kernel operator; these two operators have been compared in a variety of norms [Cox (1984), Silverman (1984)]. In this paper, we refine the existing bounds for the particular case of the spline estimator considered in Rice and Rosenblatt (1983) and its corresponding equivalent kernel estimator. We obtain detailed asymptotic expressions for the bias and covariance functions of the two estimates and provide rates of convergence. Direct comparison then shows that the two estimates are similar. They may differ somewhat in their higher order boundary behavior.

In a general setting in which prior distributions that may take on the value $\infty$ are admitted, an inference based on a posterior for a prior, $\mu$, that is "minimally compatible" with the inference is shown to have a strong property of expectation consistency, that implies a corresponding property of coherence: A nonnegative expected payoff function for a gambler's strategy is necessary 0 almost everywhere $(\mu)$. In the converse direction, under appropriate regularity conditions involving continuity of the sampling distribution and of the inference, a weaker version of coherence implies that the inference is based on a posterior distribution.

The problem of maximum likelihood estimation of Lowner ordered covariance matrices is considered. It is shown that a dual formulation of this problem is tractable and important in its own right. The interplay between the primal and dual problems suggests a general algorithm for computing the solutions to these problems. This algorithm has application to some estimation problems in balanced multivariate variance components models. The speed of convergence is also discussed for the variance components models.

Stochastic ordering between probability distributions is a widely studied concept. It arises in numerous settings and has useful applications. Since it is often easy to make value judgments when such orderings exist, it is desirable to recognize their occurrence and to model distributional structure under such orderings. Unfortunately, the necessary theory for statistical inference procedures has not been developed for many problems involving stochastic ordering and this development seems to be a difficult task. We show in this paper that the stronger notion of uniform stochastic ordering (which is equivalent to failure rate ordering for continuous distributions) is quite tractable in matters of statistical inference. In particular, we consider nonparametric maximum likelihood estimation for $k$-population problems under uniform stochastic ordering restrictions. We derive closed-form estimates even with right-censored data by a reparameterization which reduces the problem to a well-known isotonic regression problem. We also derive the asymptotic distribution of the likelihood ratio statistic for testing equality of the $k$ populations against the uniform stochastic ordering restriction. This asymptotic distribution is of the chi-bar-square type as discussed by Robertson, Wright and Dykstra. These distributional results are obtained by appealing to elegant results from empirical process theory and showing that the proposed test is asymptotically distribution free. Recurrence formulas are derived for the weights of the chi-bar-square distribution for particular cases. The theory developed in this paper is illustrated by an example involving data for survival times for carcinoma of the oropharynx.

The first anomaly in the L.R.T. for testing restricted hypotheses was observed by Warrack and Robertson. They found the L.R.T. for testing an order restriction in a normal model to be dominated by a different test. In this paper we deal with a more general situation in which the L. R. T. for testing a face of an acute cone is dominated by a different test that does not take into account some of the information in the model.

Rotationally invariant tests based on test statistics of the von Mises type are proposed under the hypothesis of spherical symmetry of a multivariate distribution. The tests are distribution-free when the hypothesis of spherical symmetry is true. The asymptotic distribution of the test statistics are derived under the null hypothesis and under any fixed alternative. A simple criterion for consistency is given. The results are illustrated by numerous examples of test statistics which give rise to tests being consistent against all alternatives.

We introduce new criteria for evaluating test statistics based on the $p$-values of the statistics. Given a set of test statistics, a good statistic is one which is robust in being reasonably sensitive to all departures from the null implied by that set. We present a constructive approach to finding the optimal statistic. We apply the criteria to two-sided problems; combining independent tests; testing that the mean of a spherical normal distribution is 0, and extensions to other spherically symmetric and exponential distributions; Bartlett's problem of testing the equality of several normal variances; and testing for one outlier in a normal linear model. For the most part, the optimal statistic is quite easy to use. Often, but not always, it is the likelihood ratio statistic.

Consider the problem of continuous invariant estimation of a distribution function with the weighted Cramer-von Mises loss. The minimaxity of the empirical distribution function, which is also the best invariant estimator, is proved for any sample size. This solves a long-standing conjecture.

Consider the problem of estimating the $p \times 1$ mean vector $\theta$ under expected squared error loss, based on the observation of two independent multivariate normal vectors $Y_1 \sim N_p(\theta, \sigma^2I)$ and $Y_2 \sim N_p(\theta, \lambda\sigma^2I)$ when $\lambda$ and $\sigma^2$ are unknown. For $p \geq 3$, estimators of the form $\delta_\eta = \eta Y_1 + (1 - \eta)Y_2$ where $\eta$ is a fixed number in (0, 1), are shown to be uniformly dominated in risk by Stein estimators in spite of the fact that independent estimates of scale are unavailable. A consequence of this result is that when $\lambda$ is assumed known, shrinkage domination is robust to incorrect specification of $\lambda$.

Sequential estimation continues observations until the observed sample satisfies a prescribed criterion. Its properties are superior on the average to those of nonsequential estimation in which the number of observations is fixed a priori. A higher-order asymptotic theory of sequential estimation is given in the framework of geometry of multidimensional curved exponential families. This gives a design principle of the second-order efficient sequential estimation procedure. It is also shown that a sequential estimation can be designed to have a covariance stabilizing effect at the same time.

The binary hypothesis testing problem of deciding between two Markov chains is formulated under memory constraints. The optimality criterion used in the exponential rate with which the probability of error approaches zero as the sample size tends to infinity. The optimal memory constrained test is shown to be the solution of a set of equations derived from suitable large deviation twistings of the transition matrices under the two hypotheses. A computational algorithm and some examples are given.

Bickel and Rosenblatt proposed a procedure for testing the goodness of fit of a specified density to observed data. The test statistic is based on the distance between the kernel density estimate and the hypothesized density, and it depends on a kernel $K$, a bandwidth $b_n$ and an arbitrary weight function $a$. We study the behavior of the asymptotic power of the test and show that a uniform kernel maximizes the power when $a > 0$.

For a given block design, an electrical network is constructed in which blocks and treatments are represented by points and observations by connections. This network has the property that the resistance between two points representing two different treatments is equal to the variance on the estimate of the corresponding treatment contrast in the usual additive block effect plus treatment effect model. This provides a simple tool for computation of contrast variances in many examples. Further applications are made to obtain lower bounds for contrast variances and upper bounds for efficiencies. A particular application is a complete solution to the problem of finding the $A$-optimal design in the case where the number of blocks equals the number of treatments and the blocks are of size 2.

The minimum aberration criterion is commonly used for selecting good fractional factorial designs. In this paper we obtain minimum aberration $2^{n - k}$ designs for $k = 3, 4$ and any $n$. For $k > 4$ analogous results are not available for general $n$ since the resolution criterion is not periodic for general $n$ and $k > 4$. However, it can be shown that for any fixed $k$, both the resolution criterion and the minimum aberration criterion have a periodicity property in $n$ for $s^{n - k}$ designs with large $n$. Furthermore, the optimal-moments criterion is periodic for any $n$ and $k$.

In this paper we present a rank analogue to Stein's two-stage procedure. We analyze its behavior to second order using existing asymptotic expansions for fixed sample size rank tests and recent results on combinations of independent rank statistics.

It is shown that, in a very general setting, the empirical likelihood method for constructing confidence intervals is Bartlett-correctable. This means that a simple adjustment for the expected value of log-likelihood ratio reduces coverage error to an extremely low $O(n^{-2})$, where $n$ denotes sample size. That fact makes empirical likelihood competitive with methods such as the bootstrap which are not Bartlett-correctable and which usually have coverage error of size $n^{-1}$. Most importantly, our work demonstrates a strong link between empirical likelihood and parametric likelihood, since the Bartlett correction had previously only been available for parametric likelihood. A general formula is given for the Bartlett correction, valid in a very wide range of problems, including estimation of mean, variance, covariance, correlation, skewness, kurtosis, mean ratio, mean difference, variance ratio, etc. The efficacy of the correction is demonstrated in a simulation study for the case of the mean.

We derive Bahadur-type representations for quantile estimates obtained from two different types of nonparametric bootstrap resampling--the commonly used uniform resampling method, where each sample value is drawn with the same probability, and importance resampling, where different sample values are assigned different resampling weights. These results are applied to obtain the relative efficiency of uniform resampling and importance resampling and to derive exact convergence rates, both weakly and strongly, for either type of resampling.

The Kalman recursion for state space models is extended to allow for likelihood evaluation and minimum mean square estimation given states with an arbitrarily large covariance matrix. The extension is computationally minor. Application is made to likelihood evaluation, state estimation, prediction and smoothing.

A predictor is a method of estimating the probability of future events over an infinite data sequence. One predictor is as strong as another if for all data sequences the former has at most the mean square error (MSE) of the latter. Given any countable set $\mathscr{D}$ of predictors, we explicitly construct a predictor $S$ that is at least as strong as every element of $\mathscr{D}$. Finite sample bounds are also given which hold uniformly on the space of all possible data.

Statistics or functions are discussed that measure agreement between certain types of partially ordered data. These poset statistics are a generalization of two familiar classes of functions: the arrangement increasing functions and the decreasing reflection functions; those functions measure agreement between linearly ordered data. Specifically, the statistics in question are functions $h(\mathbf{X}_1, \mathbf{X}_2)$ of two matrix arguments, each having $N$ rows and they measure the agreement of the ordering of the $N$ rows of the two matrices. An example is used to illustrate and motivate the discussion. One statistic in this class is applied to the example; it generalizes Wilcoxon's rank sum statistic, Spearman's rank correlation and Page's statistic for ordered alternatives.

Consider a first-order autoregressive process $X_t = \beta X_{t - 1} + \varepsilon_t$, where $\{\varepsilon_t\}$ are independent and identically distributed random errors with mean 0 and variance 1. It is shown that when $\beta = 1$ the standard bootstrap least squares estimate of $\beta$ is asymptotically invalid, even if the error distribution is assumed to be normal. The conditional limit distribution of the bootstrap estimate at $\beta = 1$ is shown to converge to a random distribution.

Under minimal conditions precise bounds are obtained for the expectation of the supremum of the weighted empirical process over the interval $(0, 1/(n(\log n)^{d - 1}))$, where $d$ is the dimension of the underlying random vectors. The allowed growth of the weight function is optimal in the iid case. The results will have broad applications in the theory of all kinds of nonstandard weighted empirical processes, such as empirical processes based on uniform spacings or $U$-statistics, where it is often not so easy to show directly (as in the iid case) that the considered suprema converge to 0 in probability.

The papers of Blackwell and Hodges (1957) and Diaconis and Graham (1981) contain an error concerning the asymptotic distribution of the number of returns to the origin of a constrained random walk. The correct distribution is given.