A new method is presented for flexible regression modeling of high dimensional data. The model takes the form of an expansion in product spline basis functions, where the number of basis functions as well as the parameters associated with each one (product degree and knot locations) are automatically determined by the data. This procedure is motivated by the recursive partitioning approach to regression and shares its attractive properties. Unlike recursive partitioning, however, this method produces continuous models with continuous derivatives. It has more power and flexibility to model relationships that are nearly additive or involve interactions in at most a few variables. In addition, the model can be represented in a form that separately identifies the additive contributions and those associated with the different multivariable interactions.

Since the pioneering work of Friedman and Stuetzle in 1981, projection-pursuit algorithms have attracted increasing attention. This is mainly due to their potential for overcoming or reducing difficulties arising in nonparametric regression models associated with the so-called curse of dimensionality, that is, the amount of data required to avoid an unacceptably large variance increasing rapidly with dimensionality. Subsequent work has, however, uncovered a dependence on dimensionality for projection-pursuit regression models. Here we propose a projection-pursuit type estimation scheme, with two additional constraints imposed, for which the rate of convergence of the estimator is shown to be independent of the dimensionality. Let $(\mathbf{X}, Y)$ be a random vector such that $\mathbf{X} = (X_1, \ldots, X_d)^T$ ranges over $R^d$. The conditional mean of $Y$ given $\mathbf{X} = \mathbf{x}$ is assumed to be the sum of no more than $d$ general smooth functions of $\beta^T_i\mathbf{x}$, where $\beta_i \in S^{d - 1}$, the unit sphere in $R^d$ centered at the origin. A least-squares polynomial spline and the final prediction error criterion are used to fit the model to a random sample of size $n$ from the distribution of $(\mathbf{X}, Y)$. Under appropriate conditions, the rate of convergence of the proposed estimator is independent of $d$.

We consider asymptotic properties of the least squares estimator (LSE) in a regression model with long-memory stationary errors. First we derive a necessary and sufficient condition that the LSE be asymptotically efficient relative to the best linear unbiased estimator (BLUE). Then we derive the asymptotic distribution of the LSE under a condition on the higher-order cumulants of the white-noise process of the errors.

Given a sample of size $n$ from a distribution $P_\lambda$, one wants to estimate a functional $\psi(\lambda)$ of the (typically infinite-dimensional) parameter $\lambda$. Lower bounds on the performance of estimators can be based on the concept of a differentiable functional $P_\lambda \rightarrow \psi(\lambda)$. In this paper we relate a suitable definition of differentiable functional to differentiability of $\alpha \rightarrow dP^{1/2}_\lambda$ and $\lambda \rightarrow \psi(\lambda)$. Moreover, we show that regular estimability of a functional implies its differentiability.

In the additive effects outliers (A.O.) model considered here one observes $Y_{j,n} = X_j + \upsilon_{j,n}, 0 \leq j \leq n$, where $\{X_j\}$ is the first order autoregressive [AR(1)] process with the autoregressive parameter $|\rho| < 1$. The A.O.'s $\{\upsilon_{j,n}, 0 \leq j \leq n\}$ are i.i.d. with distribution function (d.f.) $(1 - \gamma_n)I\lbrack x \geq 0\rbrack + \gamma_n L_n(x), x \in \mathbb{R}, 0 \leq \gamma_n \leq 1$, where the d.f.'s $\{L_n, n \geq 0\}$ are not necessarily known. This paper discusses the existence, the asymptotic normality and biases of the class of minimum distance estimators of $\rho$, defined by Koul, under the A.O. model. Their influence functions are computed and are shown to be directly proportional to the asymptotic biases. Thus, this class of estimators of $\rho$ is shown to be robust against A.O. model.

Finite-sample replacement breakdown points are derived for different types of estimators of multivariate location and covariance matrices. The role of various equivariance properties is illustrated. The breakdown point is related to a measure of performance based on large deviations probabilities. Finally, we show that one-step reweighting preserves the breakdown point.

Consider a generalized linear model with response $Y$ and scalar predictor $X$. Instead of observing $X$, a surrogate $W = X + Z$ is observed, where $Z$ represents measurement error and is independent of $X$ and $Y$. The efficient score test for the absence of association depends on $m(w) = E(X\mid W = w)$ which is generally unknown. Assuming that the distribution of $Z$ is known, asymptotically efficient tests are constructed using nonparametric estimators of $m(w)$. Rates of convergence for the estimator of $m(w)$ are established in the course of proving efficiency of the proposed test.

A relatively obscure eigenvalue due to Wielandt is used to give a simple derivation of the asymptotic distribution of the eigenvalues of a random symmetric matrix. The asymptotic distributions are obtained under a fairly general setting. An application of the general theory to the bootstrap distribution of the eigenvalues of the sample covariance matrix is given.

This paper presents the asymptotic distribution of Ihara and Kano's noniterative estimator of the uniqueness in exploratory factor analysis. When the number of factors is overestimated, the estimator is not a continuous function of the sample covariance matrix and its asymptotic distribution is not normal, but the consistency holds. It is also shown that the first-order moment of the asymptotic distribution does not exist.

Let $S_1$ and $S_2$ be two independent $p \times p$ Wishart matrices with $S_1 \sim W_p(\sum_1, n_1)$ and $S_2 \sim W_p(\sum_2, n_2)$. We wish to estimate $(\sum_1, \sum_2)$ under the loss function $L(\hat{\sum}_1, \hat{\sum}_2; \sum_1, \sum_2) = \sum_i\{\operatorname{tr}(\sum^{-1}_i \hat{\sum}_i) - \log|\sum^{-1}_i\hat{\sum}_i| - p\}$. Our approach is to first utilize the principle of invariance to narrow the class of estimators under consideration to the equivariant ones. The unbiased estimates of risk of these estimators are then computed and promising estimators are derived from them. A Monte Carlo study is also conducted to evaluate the risk performances of these estimators. The results of this paper extend those of Stein, Haff, Dey and Srinivasan from the one sample problem to the two sample one.

Let $X_1, X_2$ be two $p \times 1$ multivariate normal random vectors and $S_1, S_2$ be two $p \times p$ Wishart matrices, where $X_1 \sim N_p(\xi, \sum_1), X_2 \sim N_p(\xi, \sum_2), S_1 \sim W_p(\sum_1, n)$ and $S_2 \sim W_p(\sum_2, n)$. We further assume that $X_1, X_2, S_1, S_2$ are stochastically independent. We wish to estimate the common mean $\xi$ with respect to the loss function $L = (\hat{\xi} - \xi)'(\sum^{-1}_1 + \sum^{-1}_2)(\hat{\xi} - \xi)$. By extending the methods of Stein and Haff, an alternative unbiased estimator to the usual generalized least squares estimator is obtained. However, the risk of this estimator is not available in closed form. A Monte Carlo swindle is used instead to evaluate its risk performance. The results indicate that the alternative estimator performs very favorably against the usual estimator.

Let $X$ have a discrete density of the form $f(x) = t(x)\xi(\theta)\theta^{x_1}_1 \cdots \theta^{x_p}_p$, where $t(x)$ is nonzero on some infinite subset of $Z^p$. Consider simultaneous estimation of the $\theta_i$ under the loss $\mathscr{L}_m(\theta, \delta) = \sum^p_{i = 1} \theta^{-m}_i(\theta_i - \delta_i)^2, m \geq 0$. For integers $m \geq 1$, estimators are found which improve on the maximum likelihood estimator or uniformly minimum variance unbiased estimator. The improved estimators are distinguished by the property that they do not depend on $m$ for "large values" of the observed vector. On the other hand, we prove admissibility of a class of estimators, including the MLE and UMVUE, for some discrete densities of the indicated form under squared error loss $(m = 0)$.

This paper compares three methods for producing lower bounds on the minimax risk under quadratic loss. The first uses the bounds from Brown and Gajek. The second method also uses the information inequality and results in bounds which are always at least as good as those from the first method. The third method is the hardest-linear-family method described by Donoho and Liu. These methods are applied in four examples, the last of which relates to a frequently considered problem in nonparametric regression.

Consider i.i.d. pairs $(\theta_i, X_i), i \geq 1$, where $\theta_1$ has an unknown prior distribution $\omega$ and given $\theta_1, X_1$ has distribution $P_{\theta_1}$. This setup arises naturally in the empirical Bayes problems. We put a probability (a hyperprior) on the space of all possible $\omega$ and consider the posterior mean $\hat{\omega}$ of $\omega$. We show that, under reasonable conditions, $P_{\hat{\omega}} = \int P_\theta d\hat{\omega}$ is consistent in $L_1$. Under a identifiability assumption, this result implies that $\hat{\omega}$ is consistent in probability. As another application of the $L_1$ consistency, we consider a general empirical Bayes problem with compact state space. We prove that the Bayes empirical Bayes rules are asymptotically optimal.

The problem of finding admissible, asymptotically optimal compound rules is pursued in the infinite state case. The components involve the estimation of an arbitrary continuous transform of the natural parameter of a real exponential family with compact parameter space. We show that all Bayes estimators are admissible. Our main result is that any Bayes compound estimator versus a mixture of i.i.d. priors on the compound parameter is asymptotically optimal if the mixing hyperprior has full support. The asymptotic optimality results are generalized to weighted squared error loss with continuous weight function and applications to some nonexponential situations are also considered. Several examples of such hyperpriors are given and for some of them practically useful forms of the corresponding Bayes estimators are obtained.

Conditions are given which suffice for the assessment of a coherent inference by means of a Bayesian algorithm, i.e., a suitable extension of the classical Bayes theorem relative to a finite number of alternatives. Under some further hypotheses such inference is shown to be, in addition, coherent in the sense of Heath, Lane and Sudderth. Moreover, a characterization of coherent posteriors is provided, together with some remarks concerning finitely additive conditional probabilities.

Chaining, in combination with adaptive importance sampling, can provide an effective technique for the numerical evaluation of high-dimensional integrals in the context of a posterior analysis. In many statistical problems ways of applying chaining can be found which depend heavily on the structure of the problem. In this paper we consider a very general method of implementing chaining for arbitrary integrals. Also, we show that chaining can be applied to solve global optimization problems and prove several generalizations of a theorem of Pincus.

Three stopping rules are proposed to detect a change in a normal mean, when the initial value of the mean is unknown but an estimate obtained from a training sample is available. Asymptotic approximations are given for the average run length when there is no change. Under certain hypotheses about the length of time before the change occurs and the magnitude of the change, we obtain asymptotic approximations for the expected delay in detection in terms of the corresponding expected delay in the much simpler case of a known initial value. The results of a Monte Carlo experiment supplement our asymptotic theory to yield some general conclusions about the relative merits of the three stopping rules and guidelines for choosing the size of the training sample.

A minor modification of the product-limit estimator is proposed for estimating a distribution function (not necessarily continuous) when the data are subject to either truncation or censoring, or to both, by independent but not necessarily identically distributed truncation-censoring variables. Making use of martingale integral representations and empirical process theory, uniform strong consistency of the estimator is established and weak convergence results are proved for the entire observable range of the function. Numerical results are also given to illustrate the usefulness of the modification, particularly in the context of truncated data.

Conditions under which the bivariate Kaplan-Meier estimate of Dabrowska is not a proper survival function are given. All points assigned negative mass are identified under the assumption that the observations follow an absolutely continuous distribution. The number of points assigned negative mass increases as $n^2$ and the total amount of negative mass does not disappear as $n \rightarrow \infty$, where $n$ is the sample size. A simulation study is reported which shows that large amounts of negative mass are assigned by the estimator, amounts ranging from 0.3 to 0.6 for a sample of size 50 over a variety of parameters for bivariate exponential distributions.

Estimation of the finite population distribution function and related statistics, such as the median and interquartile range, is considered. Large-sample properties of estimators constructed from stratified cluster samples, and properties of large-sample confidence intervals, are established. The results are obtained within the context of a sequence of finite populations generated from a superpopulation.

The asymptotic accuracy of the estimated one-term Edgeworth expansion and the bootstrap approximation for a Studentized $U$-statistic is investigated. It is shown that both the Edgeworth expansion estimate and the bootstrap approximation are asymptotically closer to the exact distribution of a Studentized $U$-statistic than the normal approximation. The conditions needed to obtain these results are weak moment assumptions on the kernel $h$ of the $U$-statistic and a nonlattice condition for the distribution of $g(X_1) = E\lbrack h(X_1, X_2) \mid X_1\rbrack$. As an application improved Edgeworth and bootstrap based confidence intervals for the mean of a $U$-statistic are obtained.

An asymptotic expansion of length 2 is established for the coverage probabilities of confidence intervals for the underlying $q$-quantile which are derived by bootstrapping the sample $q$-quantile. The corresponding level error turns out to be of order $O(n^{-1/2})$ which is unexpectedly low. A confidence interval of even more practical use is derived by using backward critical points. The resulting confidence interval is of the same length as the one derived by ordinary bootstrap but it is distribution free and has higher coverage probability.

This note studies the asymptotic behavior of the delete-$d$ jackknife in the irregular case of a sample $p$-quantile, calculated from a sample of $n$ iid r.v.'s. Two results are obtained: (a) an almost sure rate of convergence of the delete-$d$ jackknife histogram to the normal distribution; (b) almost sure convergence of the delete-$d$ jackknife variance estimate to the asymptotic variance of the sample $p$-quantile.