Many statisticians have adopted compact differentiability since Reeds showed in 1976 that it holds (while Frechet differentiability fails) in the supremum (sup) norm on the real line for the inverse operator and for the composition operator $(F,G) \mapsto F \circ G$ with respect to $F$. However, these operators are Frechet differentiable with respect to $p$-variation norms, which for $p > 2$ share the good probabilistic properties of the sup norm, uniformly over all distributions on the line. The remainders in these differentiations are of order $\| \cdot \|^\gamma$ for $\gamma > 1$. In a range of cases $p$-variation norms give the largest possible values of $\gamma$ on spaces containing empirical distribution functions, for both the inverse and composition operators. Compact differentiability in the sup norm cannot provide such remainder bounds since, over some compact sets, differentiability holds arbitrarily slowly.

Bartlett type identities are shown to exist for martingales. As applications, we give a cumulant-based proof of the martingale central limit theorem, and we give an algorithm for calculating approximate cumulants of the least squares estimator in the AR(1) process.

Hajek established a local asymptotic minimax risk bound for appropriate symmetric loss functions and also gave a necessary condition for the risk of an estimator to attain the lower bound. We extend these results to the case of asymmetric loss functions. The asymmetry brings about the shift of location of the loss functions. Besides, the optimal estimator that attains the bound is shown to have asymptotic normal distribution with asymptotic bias.

A unified approach to deriving confidence regions in linear functional relationship models is presented, based on the conditional likelihood ratio method of Knowles, Siegmund and Zhang. In the case of a single latent predictor, the confidence region for the slope produced by this approach is the familiar one of Fieller and Creasy. However, here it is shown how to derive a confidence region for the slope, when it is known that the slope is positive, that improves on merely intersecting the region for an unrestricted slope with $(0,\infty)$. A geometric interpretation is given for Fieller-Creasy confidence region for the ratio of population means (Fieller-Creasy problem). Regions are also derived for simultaneous estimation of the slope and intercept in the model with a single latent predictor, and for the slopes in a model with two latent predictors.

This article deals with the construction of confidence intervals when the components of the location parameter $\mu$ of the random variable $\mathbf{X}$, which is elliptically symmetrically distributed, are subject to order restrictions. Several domination results are proved by studying the derivative of the coverage probability of the confidence intervals centered at the improved point estimators. Consequently, we strengthen the previously known results regarding the simple ordering and obtain several new results for other general forms of order restrictions, including the simple tree ordering, the umbrella ordering, the simple and the double loop ordering and some combination of these. These domination results are obtained under the assumption that $\Sigma$ is a diagonal matrix. When $\Sigma$ is nondiagonal, some new intervals are introduced which dominate the standard intervals centered at the unrestricted maximum likelihood estimator for various types of order restrictions. Similar results are obtained for scale parameters as well. Contrary to the location problems, in case of the scale parameters satisfying the simple ordering we find that the restricted maximum likelihood estimator of the largest parameter fails to universally dominate the unrestricted maximum likelihood estimator. A similar negative result is noted for simple tree order restriction.

In the multivariate calibration problem using a multivariate linear model, an exact confidence region is constructed. It is shown that the region is always nonempty and is invariant under nonsingular transformations.

A simple topological criterion is given for the existence of a sequence of tests for composite hypothesis testing problems, such that almost surely only finitely many errors are made.

Let $X_1, \ldots, X_M, Y_1,\ldots, Y_N$ be random variables, and set $\mathbf{X} = (X_1, \ldots, X_M)$ and $\mathbf{Y} = (Y_1, \ldots, Y_N)$. Let $\varphi$ be the regression or logistic or Poisson regression function of $\mathbf{Y}$ on $\mathbf{X}(N = 1)$ or the logarithm of the density function of $\mathbf{Y}$ or the conditional density function of $\mathbf{Y}$ on $\mathbf{X}$. Consider the approximation $\varphi^\ast$ to $\varphi$ having a suitably defined form involving a specified sum of functions of at most $d$ of the variables $x_1, \ldots, x_M, y_1,\ldots, y_N$ and, subject to this form, selected to minimize the mean squared error of approximation or to maximize the expected log-likelihood or conditional log-likelihood, as appropriate, given the choice of $\varphi$. Let $p$ be a suitably defined lower bound to the smoothness of the components of $\varphi^\ast$. Consider a random sample of size $n$ from the joint distribution of $\mathbf{X}$ and $\mathbf{Y}$. Under suitable conditions, the least squares or maximum likelihood method is applied to a model involving nonadaptively selected sums of tensor products of polynomial splines to construct estimates of $\varphi^\ast$ and its components having the $L_2$ rate of convergence $n^{-p/(2p + d)}$.

A modification of kernel density estimation is proposed. The first step is ordinary kernel estimation of the density and its cdf. In the second step the data are transformed, using this estimated cdf, to an approximate uniform (or normal or other target) distribution. The density and cdf of the transformed data are then estimated by the kernel method and, by change of variable, converted to new estimates of the density and the cdf of the original data. This process is repeated for a total of $k$ steps for some integer $k$ greater than 1. If the target density is uniform, then the order of the bias is reduced, provided that the density of the observed data is sufficiently smooth. By proper choice of bandwidth, rates of squared-error convergence equal to those of higher-order kernels are attainable. More precisely, $k$ repetitions of the process are equivalent, in terms of rate of convergence, to a $2k$-th-order kernel. This transformation-kernel estimate is always a bona fide density and appears to be more effective at small sample sizes than higher-order kernel estimators, at least for densities with interesting features such as multiple modes. The main theoretical achievement of this paper is the rigorous establishment of rates of convergence under multiple iteration. Simulations using a uniform target distribution suggest that the possibility of improvement over ordinary kernel estimation is of practical significance for samples sizes as low as 100 and can become appreciable for sample sizes around 400.

Chen and Shiau showed that a two-stage spline smoothing method and the partial regression method lead to efficient estimators for the parametric component of a partially linear model when the smoothing parameter is a deterministic sequence tending to zero at an appropriate rate. This paper is concerned with the large-sample behavior of these estimators when the smoothing parameter is chosen by the generalized cross validation (GCV) method or Mallows' $C_L$. Under mild conditions, the estimated parametric component is asymptotically normal with the usual parametric rate of convergence for both spline estimation methods. As a by-product, it is shown that the "optimal rate" for the smoothing parameter, with respect to expected average squared error, is the same for the two estimation methods as it is for ordinary smoothing splines.

New formulas are given for the minimax linear risk in estimating a linear functional of an unknown object from indirect data contaminated with random Gaussian noise. The formulas cover a variety of loss functions and do not require the symmetry of the convex a priori class. It is shown that affine minimax rules are within a few percent of minimax even among nonlinear rules, for a variety of loss functions. It is also shown that difficulty of estimation is measured by the modulus of continuity of the functional to be estimated. The method of proof exposes a correspondence between minimax affine estimates in the statistical estimation problem and optimal algorithms in the theory of optimal recovery.

Mallows has conjectured that among distributions which are Gaussian but for occasional contamination by additive noise, the one having least Fisher information has (two-sided) geometric contamination. A very similar problem arises in estimation of a nonnegative vector parameter in Gaussian white noise when it is known also that most [i.e., $(1 - \varepsilon)$] components are zero. We provide a partial asymptotic expansion of the minimax risk as $\varepsilon \rightarrow 0$. While the conjecture seems unlikely to be exactly true for finite $\varepsilon$, we verify it asymptotically up to the accuracy of the expansion. Numerical work suggests the expansion is accurate for $\varepsilon$ as large as 0.05. The best $l_1$-estimation rule is first- but not second-order minimax. The results bear on an earlier study of maximum entropy estimation and various questions in robustness and function estimation using wavelet bases.

In the point and interval estimation of the variance of a normal distribution with an unknown mean, the best affine equivariant estimators are dominated by Stein's truncated and Brewster and Zidek's smooth procedures, which are separately derived. This paper gives a unified approach to this problem by using a simple definite integral and provides a class of improved procedures in both point and interval estimation of powers of the scale parameter of normal, lognormal, exponential and Pareto distributions. Finally, the same method is applied to the improvement on the James-Stein rule in the simultaneous estimation of a multinormal mean.

For some time, so-called empirical likelihoods have been used heuristically for purposes of nonparametric estimation. Owen showed that empirical likelihood ratio statistics for various parameters $\theta(F)$ of an unknown distribution $F$ have limiting chi-square distributions and may be used to obtain tests or confidence intervals in a way that is completely analogous to that used with parametric likelihoods. Our objective in this paper is twofold: first, to link estimating functions or equations and empirical likelihood; second, to develop methods of combining information about parameters. We do this by assuming that information about $F$ and $\theta$ is available in the form of unbiased estimating functions. Empirical likelihoods for parameters are developed and shown to have properties similar to those for parametric likelihood. Efficiency results for estimates of both $\theta$ and $F$ are obtained. The methods are illustrated on several problems, and areas for future investigation are noted.

The fundamental result on the rate of convergence of the EM algorithm has proven to be theoretically valuable and practically useful. Here, this result is generalized to the ECM algorithm, a more flexible and applicable iterative algorithm proposed recently by Meng and Rubin. Results on the rate of convergence of variations of ECM are also presented. An example is given to show that intuitions accurate for complete-data iterative algorithms may not be trustworthy in the presence of missing data.

A quasiscore function, as defined by Wedderburn and by McCullagh, frequently fails to have a symmetric derivative matrix. Such a score function cannot be the gradient of any potential function on the parameter space; that is, there is no "quasilikelihood." Without a likelihood function it is difficult to distinguish good roots from bad roots or to set satisfactory confidence limits. From a different perspective, a potential function seems to be essential in order to give the theory an approximate Bayesian interpretation. The purpose of this paper is to satisfy these needs by developing a method of projecting the true score function onto a class of conservative estimating functions. By construction, a potential function for the projected score exists having many properties of a log-likelihood function.

We consider the Hotelling $T^2$ statistic for an arbitrary $d$-dimensional sample. If the sampling is not too deterministic or inhomogeneous, then under the zero-means hypothesis the limiting distribution for $T^2$ is $\chi^2_d$. It is shown that a test for the orthant symmetry condition introduced by Efron can be constructed which does not differ essentially from the one based on $\chi^2_d$ and at the same time is applicable not only to large random homogeneous samples but to all multidimensional samples. The main results are not limit theorems, but exact inequalities corresponding to the solutions to certain extremal problems. The following auxiliary result itself may be of interest: $\chi_d - \sqrt{d - 1}$ has a monotone likelihood ratio.

A class of exponential transformation models is defined on symmetric cones $\Omega$ with the group of automorphisms on $\Omega$ as the acting group. We show that these models are reproductive and the exponent of their joint distribution for a given sample of size $q$ can be split into $q$ independent components, introducing one sample point at a time. The automorphism group can be factorized into the group of positive dilation and another group. Accordingly, the symmetric cone $\Omega$ can be seen as the direct product of $\mathbb{R}^+$ and a unit orbit, and every $x$ in $\Omega$ can be identified by its orbital decomposition. We derive the distributions of the independent components of the exponent, of the "length" of $x$, the "direction" of $x$, the conditional distribution of the direction given the length and other distributions for a given sample. The Wishart distribution and the hyperboloid distribution are two special cases in the class we define. We also give a unified view of several distributions which are usually treated separately.

This paper extends the study of Wishart and multivariate beta distributions to the singular case, where the rank is below the dimensionality. The usual conjugacy is extended to this case. A volume element on the space of positive semidefinite $m \times m$ matrices of rank $n < m$ is introduced and some transformation properties established. The density function is found for all rank-$n$ Wishart distributions as well as the rank-1 multivariate beta distribution. To do that, the Jacobian for the transformation to the singular value decomposition of general $m \times n$ matrices is calculated. The results in this paper are useful in particular for updating a Bayesian posterior when tracking a time-varying variance-covariance matrix.

We solve the following variational problem: Find the maximum of $E\|X - Y\|$ subject to $E\|X\|^2 \leq 1$, where $X$ and $Y$ are i.i.d. random $n$-vectors, and $\|\cdot\|$ is the usual Euclidean norm on $\mathbb{R}^n$. This problem arose from an investigation into multidimensional scaling, a data analytic method for visualizing proximity data. We show that the optimal $X$ is unique and is (1) uniform on the surface of the unit sphere, for dimensions $n \geq 3$, (2) circularly symmetric with a scaled version of the radial density $\rho/(1 - \rho^2)^{1/2}, 0 \leq \rho \leq 1$, for $n = 2$, and (3) uniform on an interval centered at the origin, for $n = 1$ (Plackett's theorem). By proving spherical symmetry of the solution, a reduction to a radial problem is achieved. The solution is then found using the Wiener-Hopf technique for (real) $n < 3$. The results are reminiscent of classical potential theory, but they cannot be reduced to it. Along the way, we obtain results of independent interest: for any i.i.d. random $n$-vectors $X$ and $Y, E\|X - Y\| \leq E\|X + Y\|$. Further, the kernel $K_{p,\beta}(x, y) = \|x + y\|^\beta_p - \|x - y\|^\beta_p, x, y \in \mathbb{R}^n$ and $\|x\|p = (\sum|x_i|^p)^{1/p}$, is positive-definite, that is, it is the covariance of a random field, $K_{p,\beta}(x, y) = E\lbrack Z(x)Z(y)\rbrack$ for some real-valued random process $Z(x)$, for $1 \leq p \leq 2$ and $0 < \beta \leq p \leq 2$ (but not for $\beta > p$ or $p > 2$ in general). Although this is an easy consequence of known results, it appears to be new in a strict sense. In the radial problem, the average distance $D(r_1, r_2)$ between two spheres of radii $r_1$ and $r_2$ is used as a kernel. We derive properties of $D(r_1, r_2)$, including nonnegative definiteness on signed measures of zero integral.

Maximal inequalities for degenerate $U$-processes of order $k, k \geq 1$, are established. The results rest on a moment inequality (due to Bonami) for $k$th-order forms and on extensions of chaining and symmetrization inequalities from the theory of empirical processes. Rates of uniform convergence are obtained. The maximal inequalities can be used to determine the limiting distribution of estimators that optimize criterion functions having $U$-process structure. As an application, a semiparametric regression estimator that maximizes a $U$-process of order 3 is shown to be $\sqrt n$-consistent and asymptotically normally distributed.

We introduce a general class of conditional $U$-statistics and present sufficient conditions for their universal consistency in $r$th mean. It is shown that under mild assumptions on the smoothing parameters, window and $k_n$-nearest neighbor estimators are universally consistent. An application to a new nonparametric discrimination problem is also included.

If $R_n$ is the range from an exchangeable sequence of random variables, then $\{R_n/n\}$ forms a reverse submartingale sequence.

For a random sample of size $n$ from an absolutely continuous bivariate population $(X, Y)$, let $X_{i:n}$ denote the $i$th order statistic of the $X$ sample values. The $Y$-value associated with $X_{i:n}$ is denoted by $Y_{\lbrack i:n\rbrack}$ and is called the concomitant of the $i$th order statistic. For $1 \leq k \leq n$, let $V_{k,n} = \max(Y_{\lbrack n - k + 1: n\rbrack},\ldots,Y_{\lbrack n: n\rbrack})$. In this paper, we discuss the finite-sample and the asymptotic distributions of $V_{k,n}$. We investigate the limit distribution of $V_{k,n}$ as $n \rightarrow \infty$, when $k$ is held fixed and when $k = \lbrack np\rbrack, 0 < p < 1$. In both cases we obtain simple sufficient conditions and determine the associated norming constants. We apply our results to some interesting situations, including the bivariate normal population and the simple linear regression model.

A new approach to the problem of identifying a nonlinear time series model is considered. We argue that cumulative lagged conditional mean and variance functions are the appropriate "signatures" of a nonlinear time series for the purpose of model identification, being analogous to cumulative distribution functions or cumulative hazard functions in iid models. We introduce estimators of the cumulative lagged conditional mean and variance functions and study their asymptotic properties. A goodness-of-fit test for parametric time series models is also developed.

We study problems of semiparametric statistical inference connected with long-memory covariance stationary time series, having spectrum which varies regularly at the origin: There is an unknown self-similarity parameter, but elsewhere the spectrum satisfies no parametric or smoothness conditions, it need not be in $L_p$, for any $p > 1$, and in some circumstances the slowly varying factor can be of unknown form. The basic statistic of interest is the discretely averaged periodogram, based on a degenerating band of frequencies around the origin. We establish some consistency properties under mild conditions. These are applied to show consistency of new estimates of the self-similarity parameter and scale factor. We also indicate applications of our results to standard errors of least squares estimates of polynomial regression with long-memory errors, to generalized least squares estimates of this model and to estimates of a "cointegrating" relationship between long-memory time series.

This paper establishes the uniform closeness of a randomly weighted residual empirical process to its natural estimator via weak convergence techniques. The weights need not be independent, bounded or even square integrable. This result is used to yield the asymptotic uniform linearity of a class of rank statistics in $p$th-order autoregression models. The latter result, in turn, yields the asymptotic distributions of a class of robust and Jaeckel-type rank estimators. The main result is also used to obtain the asymptotic distributions of the least absolute deviation and certain other robust minimum distance estimators of the autoregression parameter vector.