A new method for smoothing a multivariate data set is introduced that is based on a simple geometric operation. This method is applied to the problem of estimating level sets of a density and minimum volume sets with given probability content. Building on existing techniques, the resulting estimator combines excellent theoretical and computational properties: It converges with the minimax rates (up to log factors) in most cases where these rates are known and, at the same time, it can be computed, visualized, stored and manipulated by simple algorithms and tools. It is applicable to a wide class of sets that is characterized explicitly in terms of the underlying densities and includes nonconvex and disconnected sets, and it is argued that it should give reasonable results in completely general situations. Applications to the construction of multivariate confidence regions in frequentist and Bayesian contexts are briefly mentioned.

We suggest a new approach, based on the use of density estimators, for the problem of estimating the (compact) support of a multivariate density. This subject (motivated in terms of pattern analysis by Grenander) has interesting connections with detection and clustering.

A natural class of density-based estimators is defined. Universal consistency results and convergence rates are established for these estimators, with respect to the usual measure-based metric $d_{\mu}$ between sets. Further convergence rates (with respect to both $d_{\mu}$ and the Hausdorff metric $d_H$) are also obtained under some, fairly intuitive, shape restrictions.

A theory of optimum orthogonal fractions is developed for Fourier regression models using integer lattice designs. These provide alternatives to simple grids (product designs) in the case when specified main effects and interaction terms are required to be analyzed. The challenge is to obtain sample sizes which are polynomial in the dimension rather than exponential. This is achieved for certain models with special algorithms based on both algebraic generation and more direct sequential search.

In approximate design theory, necessary and sufficient conditions that a repeated measurements design be universally optimal are given as linear equations whose unknowns are the proportions of subjects on the treatment sequences. Both the number of periods and the number of treatments in the designs are arbitrary, as is the covariance matrix of the normal response model. The existence of universally optimal "symmetric" designs is proved; the single linear equation which the proportions satisfy is given. A formula for the information matrix of a universally optimal design is derived.

It is shown that the standard two one-sided tests procedure for bioequivalence is a biased test. Better tests exist. In this paper, an unbiased $\alpha$-level test and other tests which are uniformly more powerful than the two one-sided tests procedure are constructed. Its power can be noticeably larger than that of the $\alpha$-level two one-sided tests procedure.

We study the problem of testing a simple null hypothesis about the multivariate normal mean vector against smooth or piecewise smooth cone alternatives. We show that the mixture weights of the $\bar{\chi}^2$ distribution of the likelihood ratio test can be characterized as mixed volumes of the cone and its dual. The weights can be calculated by integration involving the second fundamental form on the boundary of the cone. We illustrate our technique by examples involving a spherical cone and a piecewise smooth cone.

A general way of constructing classes of goodness-of-fit tests for multivariate samples is presented. These tests are based on a random signed measure that plays the same role as the empirical process in the construction of the classical Kolmogorov-Smirnov tests. The resulting tests are consistent against any fixed alternative, and, for each sequence of contiguous alternatives, a test in each class can be chosen so as to optimize the discrimination of those alternatives.

By noting the connection with k-sample U-statistics, we find a simple decomposition of the variance of the cross-match estimate, which can be regarded as a generalization of Efron and Stein. We apply the decomposition in assessing efficiencies of several plans of using the weighted samples from an importance scheme. The applications of the formula to multiple imputations lead to a method of crossing jointly imputed data to gain more accuracy.

Suppose that there are $k \geq 2$ different systems (i.e., stochastic processes), where each system has an unknown steady-state mean performance and unknown asymptotic variance. We allow for the asymptotic variances to be unequal and for the distributions of the k systems to be different. We consider the problem of running independent, single-stage simulations to make multiple comparisons of the steady-state means of the different systems. We derive asymptotically valid (as the run lengths of the simulations of the systems tend to infinity) simultaneous confidence intervals for each of the following problems: all pairwise comparisons of means, all contrasts, multiple comparisons with a control and multiple comparisons with the best. Our confidence intervals are based on standardized time series methods, and we establish the asymptotic validity of each under the sole assumption that the stochastic processes representing the simulation output of the different systems satisfy a functional central limit theorem. Although simulation is the context of this paper, the results naturally apply to (asymptotically) stationary time series.

Assume ${P_{\theta}: \theta \epsilon \Theta}$ is a set of probability distributions with a common dominating measure on a complete separable metric space Y. A state $\theta^* \epsilon \Theta$ is chosen by Nature. A statistician obtains n independent observations $Y_1, \dots, Y_n$ from Y distributed according to $P_{\theta^*}$. For each time t between 1 and n, based on the observations $Y_1, \dots, Y_{t-1}$, the statistician produces an estimated distribution $\hat{P}_t$ for $P_{\theta^*}$ and suffers a loss $L(P_{\theta^*}, \hat{P}_t)$. The cumulative risk for the statistician is the average total loss up to time n. Of special interest in information theory, data compression, mathematical finance, computational learning theory and statistical mechanics is the special case when the loss $L(P_{\theta^*}, \hat{P}_t)$ is the relative entropy between the true distribution $P_{\theta^*}$ and the estimated distribution $\hat{P}_t$. Here the cumulative Bayes risk from time 1 to n is the mutual information between the random parameter $\Theta^*$ and the observations $Y_1, \dots, Y_n$.

New bounds on this mutual information are given in terms of the Laplace transform of the Hellinger distance between pairs of distributions indexed by parameters in $\Theta$. From these, bounds on the cumulative minimax risk are given in terms of the metric entropy of $\Theta$ with respect to the Hellinger distance. The assumptions required for these bounds are very general and do not depend on the choice of the dominating measure. They apply to both finite- and infinite-dimensional $\Theta$. They apply in some cases where Y is infinite dimensional, in some cases where Y is not compact, in some cases where the distributions are not smooth and in some parametric cases where asymptotic normality of the posterior distribution fails.

$L_1$-optimal minimum distance estimators are provided for a projection pursuit regression type function with smooth functional components that are either additive or multiplicative, in the presence of or without interactions. The obtained rates of convergence of the estimate to the true parameter depend on Kolmogorov's entropy of the assumed model and confirm Stone's heuristic dimensionality reduction principle. Rates of convergence are also obtained for the error in estimating the derivatives of a regression type function.

The problem of optimal adaptive estimation of a function at a given point from noisy data is considered. Two procedures are proved to be asymptotically optimal for different settings.

First we study the problem of bandwidth selection for nonparametric pointwise kernel estimation with a given kernel. We propose a bandwidth selection procedure and prove its optimality in the asymptotic sense. Moreover, this optimality is stated not only among kernel estimators with a variable bandwidth. The resulting estimator is asymptotically optimal among all feasible estimators. The important feature of this procedure is that it is fully adaptive and it "works" for a very wide class of functions obeying a mild regularity restriction. With it the attainable accuracy of estimation depends on the function itself and is expressed in terms of the "ideal adaptive bandwidth" corresponding to this function and a given kernel.

The second procedure can be considered as a specialization of the first one under the qualitative assumption that the function to be estimated belongs to some Hölder class $\Sigma (\beta, L)$ with unknown parameters $\beta, L$. This assumption allows us to choose a family of kernels in an optimal way and the resulting procedure appears to be asymptotically optimal in the adaptive sense in any range of adaptation with $\beta \leq 2$.

An inequality is given for the expected length of a confidence interval given that a particular distribution generated the data and assuming that the confidence interval has a given coverage probability over a family of distributions. As a corollary, attempts to adapt to the regularity of the true density within derivative smoothness classes cannot improve the rate of convergence of the length of the confidence interval over minimax fixed-length intervals and still maintain uniform coverage probability. However, adaptive confidence intervals can attain improved rates of convergence in some other classes of densities, such as those satisfying a shape restriction.

We develop a general theory which provides a unified treatment for the asymptotic normality and efficiency of the maximum likelihood estimates (MLE's) in parametric, semiparametric and nonparametric models. We find that the asymptotic behavior of substitution estimates for estimating smooth functionals are essentially governed by two indices: the degree of smoothness of the functional and the local size of the underlying parameter space. We show that when the local size of the parameter space is not very large, the substitution standard (nonsieve), substitution sieve and substitution penalized MLE's are asymptotically efficient in the Fisher sense, under certain stochastic equicontinuity conditions of the log-likelihood. Moreover, when the convergence rate of the estimate is slow, the degree of smoothness of the functional needs to compensate for the slowness of the rate in order to achieve efficiency. When the size of the parameter space is very large, the standard and penalized maximum likelihood procedures may be inefficient, whereas the method of sieves may be able to overcome this difficulty. This phenomenon is particularly manifested when the functional of interest is very smooth, especially in the semiparametric case.

Given noisy data, function estimation is considered when the unknown function is known a priori to be either convex or concave on each of a small number of regions where the function. When the number of regions is unknown, the model selection problem is to determine the number of convexity change points. For kernel estimates in Gaussian noise, the number of false change points is evaluated as a function of the smoothing parameter. To insure that the number of false convexity change points tends to zero, the smoothing level must be larger than is generically optimal for minimizing the mean integrated square error (MISE). A two-stage estimator is proposed and shown to achieve the optimal rate of convergence of the MISE. In the first stage, the number and location of the change points is estimated using strong smoothing. In the second stage, a constrained smoothing spline fit is performed with the smoothing level chosen to minimize the MISE. The imposed constraint is that a single change point occur in a region about each empirical change point from the first-stage estimate. This constraint is equivalent to the requirement that the third derivative of the second-stage estimate has a single sign in a small neighborhood about each first-stage change point. The change points from the second stage are first-stage change points, but need not be at the identical locations.

Fixed parameter asymptotic statements are often used in the context of nonparametric curve estimation problems (e.g., nonparametric density or regression estimation). In this context several forms of superefficiency can occur. In contrast to what can happen in regular parametric problems, here every parameter point (e.g., unknown density or regression function) can be a point of superefficiency.

We begin with an example which shows how fixed parameter asymptotic statements have often appeared in the study of adaptive kernel estimators, and how superefficiency can occur in this context. We then carry out a more systematic study of such fixed parameter statements. It is shown in four general settings how the degree of superefficiency attainable depends on the structural assumptions in each case.

We introduce a method to select a smoothing factor for kernel density estimation such that, for all densities in all dimensions, the $L_1$ error of the corresponding kernel estimate is not larger than three times the error of the estimate with the optimal smoothing factor plus a constant times $\sqrt{\log n/n}$, where n is the sample size, and the constant depends only on the complexity of the kernel used in the estimate. The result is nonasymptotic, that is, the bound is valid for each n. The estimate uses ideas from the minimum distance estimation work of Yatracos. As the inequality is uniform with respect to all densities, the estimate is asymptotically minimax optimal (modulo a constant) over many function classes.

The M-estimators are proposed for the linear regression model with random design when the response observations are doubly censored. The proposed estimators are constructed as some functional of a Campbell-type estimator $\hat{F}_n$ for a bivariate distribution function based on data which are doubly censored in one coordinate. We establish strong uniform consistency and asymptotic normality of $\hat{F}_n$ and derive the asymptotic normality of the proposed regression M-estimators through verifying their Hadamard differentiability property. As corollaries, we show that our results on the proposed M-estimators also apply to other types of data such as uncensored observations, bivariate observations under univariate right censoring, bivariate right-censored observations, and so on. Computation of the proposed regression M-estimators is discussed and the method is applied to a doubly censored data set, which was encountered in a recent study on the age-dependent growth rate of primary breast cancer.

Pertaining to the estimating equations proposed by Tsiatis, based on the linear rank test, we show the existence of local confounding between the baseline hazard function and the covariates. Due to the local confounding, an estimating equation in Tsiatis' family with a larger time point of truncation could contain less information about the regression parameter than the estimating equation with a smaller time point of truncation. This phenomenon further indicates significant loss of efficiency of Tsiatis' estimating equations as well as the power loss of log-rank type tests when the baseline hazard function goes up and down along the time scale. To take care of this local confounding without using nonparametric estimates of the derivative of the baseline hazard function, we propose the empirical process approach (EPA) based on an empirical process constructed from Tsiatis' log-rank estimating equation by varying its truncating time point. The EPA will provide very tractable estimations of the regression parameters as well as Pearson's chi-squared statistics for testing the model's assumptions. Specifically, the performance of the EPA estimator is shown to be very close to the best estimator in Tsiatis' family.

We propose simple estimators for the transformation function $\Lambda$ and the distribution function F of the error for the model $$\Lambda (Y) = \alpha + \mathbf{X} \mathbf{\beta} + \varepsilon.$$ It is proved that these estimators are consistent and can achieve the unusual $n^{-1/2}$ rate of convergence on any finite interval under some regularity conditions. We show that our estimators are more attractive than another class of estimators proposed by Horowitz. Interesting decompositions of the estimators are obtained. The estimator of F is independent of the unknown transformation function $\Lambda$, and the variance of the estimator for $\Lambda$ depends on $\Lambda$ only through the density function of X. Through simulations, we find that the procedure is not sensitive to the choice of bandwidth, and the computation load is very modest. In almost all cases simulated, our procedure works substantially better than median nonparametric regression.