We consider a wavelet thresholding approach to adaptive variance function estimation in heteroscedastic nonparametric regression. A data-driven estimator is constructed by applying wavelet thresholding to the squared first-order differences of the observations. We show that the variance function estimator is nearly optimally adaptive to the smoothness of both the mean and variance functions. The estimator is shown to achieve the optimal adaptive rate of convergence under the pointwise squared error simultaneously over a range of smoothness classes. The estimator is also adaptively within a logarithmic factor of the minimax risk under the global mean integrated squared error over a collection of spatially inhomogeneous function classes. Numerical implementation and simulation results are also discussed.

In this paper we develop a nonparametric regression method that is simultaneously adaptive over a wide range of function classes for the regression function and robust over a large collection of error distributions, including those that are heavy-tailed, and may not even possess variances or means. Our approach is to first use local medians to turn the problem of nonparametric regression with unknown noise distribution into a standard Gaussian regression problem and then apply a wavelet block thresholding procedure to construct an estimator of the regression function. It is shown that the estimator simultaneously attains the optimal rate of convergence over a wide range of the Besov classes, without prior knowledge of the smoothness of the underlying functions or prior knowledge of the error distribution. The estimator also automatically adapts to the local smoothness of the underlying function, and attains the local adaptive minimax rate for estimating functions at a point.

A key technical result in our development is a quantile coupling theorem which gives a tight bound for the quantile coupling between the sample medians and a normal variable. This median coupling inequality may be of independent interest.

This paper provides ANOVA inference for nonparametric local polynomial regression (LPR) in analogy with ANOVA tools for the classical linear regression model. A surprisingly simple and exact local ANOVA decomposition is established, and a local R-squared quantity is defined to measure the proportion of local variation explained by fitting LPR. A global ANOVA decomposition is obtained by integrating local counterparts, and a global R-squared and a symmetric projection matrix are defined. We show that the proposed projection matrix is asymptotically idempotent and asymptotically orthogonal to its complement, naturally leading to an F-test for testing for no effect. A by-product result is that the asymptotic bias of the “projected” response based on local linear regression is of quartic order of the bandwidth. Numerical results illustrate the behaviors of the proposed R-squared and F-test. The ANOVA methodology is also extended to varying coefficient models.

When using the bootstrap in the presence of measurement error, we must first estimate the target distribution function; we cannot directly resample, since we do not have a sample from the target. These and other considerations motivate the development of estimators of distributions, and of related quantities such as moments and quantiles, in errors-in-variables settings. We show that such estimators have curious and unexpected properties. For example, if the distributions of the variable of interest, W, say, and of the observation error are both centered at zero, then the rate of convergence of an estimator of the distribution function of W can be slower at the origin than away from the origin. This is an intrinsic characteristic of the problem, not a quirk of particular estimators; the property holds true for optimal estimators.

The kth-nearest neighbor rule is arguably the simplest and most intuitively appealing nonparametric classification procedure. However, application of this method is inhibited by lack of knowledge about its properties, in particular, about the manner in which it is influenced by the value of k; and by the absence of techniques for empirical choice of k. In the present paper we detail the way in which the value of k determines the misclassification error. We consider two models, Poisson and Binomial, for the training samples. Under the first model, data are recorded in a Poisson stream and are “assigned” to one or other of the two populations in accordance with the prior probabilities. In particular, the total number of data in both training samples is a Poisson-distributed random variable. Under the Binomial model, however, the total number of data in the training samples is fixed, although again each data value is assigned in a random way. Although the values of risk and regret associated with the Poisson and Binomial models are different, they are asymptotically equivalent to first order, and also to the risks associated with kernel-based classifiers that are tailored to the case of two derivatives. These properties motivate new methods for choosing the value of k.

A class of estimators of the Rényi and Tsallis entropies of an unknown distribution f in ℝ^m is presented. These estimators are based on the kth nearest-neighbor distances computed from a sample of N i.i.d. vectors with distribution f. We show that entropies of any order q, including Shannon’s entropy, can be estimated consistently with minimal assumptions on f. Moreover, we show that it is straightforward to extend the nearest-neighbor method to estimate the statistical distance between two distributions using one i.i.d. sample from each.

Given a finite collection of estimators or classifiers, we study the problem of model selection type aggregation, that is, we construct a new estimator or classifier, called aggregate, which is nearly as good as the best among them with respect to a given risk criterion. We define our aggregate by a simple recursive procedure which solves an auxiliary stochastic linear programming problem related to the original nonlinear one and constitutes a special case of the mirror averaging algorithm. We show that the aggregate satisfies sharp oracle inequalities under some general assumptions. The results are applied to several problems including regression, classification and density estimation.

In the popular approach of “Bayesian variable selection” (BVS), one uses prior and posterior distributions to select a subset of candidate variables to enter the model. A completely new direction will be considered here to study BVS with a Gibbs posterior originating in statistical mechanics. The Gibbs posterior is constructed from a risk function of practical interest (such as the classification error) and aims at minimizing a risk function without modeling the data probabilistically. This can improve the performance over the usual Bayesian approach, which depends on a probability model which may be misspecified. Conditions will be provided to achieve good risk performance, even in the presence of high dimensionality, when the number of candidate variables “K” can be much larger than the sample size “n.” In addition, we develop a convenient Markov chain Monte Carlo algorithm to implement BVS with the Gibbs posterior.

The generalized varying coefficient partially linear model with a growing number of predictors arises in many contemporary scientific endeavor. In this paper we set foot on both theoretical and practical sides of profile likelihood estimation and inference. When the number of parameters grows with sample size, the existence and asymptotic normality of the profile likelihood estimator are established under some regularity conditions. Profile likelihood ratio inference for the growing number of parameters is proposed and Wilk’s phenomenon is demonstrated. A new algorithm, called the accelerated profile-kernel algorithm, for computing profile-kernel estimator is proposed and investigated. Simulation studies show that the resulting estimates are as efficient as the fully iterative profile-kernel estimates. For moderate sample sizes, our proposed procedure saves much computational time over the fully iterative profile-kernel one and gives stabler estimates. A set of real data is analyzed using our proposed algorithm.

For a random matrix following a Wishart distribution, we derive formulas for the expectation and the covariance matrix of compound matrices. The compound matrix of order m is populated by all m×m-minors of the Wishart matrix. Our results yield first and second moments of the minors of the sample covariance matrix for multivariate normal observations. This work is motivated by the fact that such minors arise in the expression of constraints on the covariance matrix in many classical multivariate problems.

Robust estimators of location and dispersion are often used in the elliptical model to obtain an uncontaminated and highly representative subsample by trimming the data outside an ellipsoid based in the associated Mahalanobis distance. Here we analyze some one (or k)-step Maximum Likelihood Estimators computed on a subsample obtained with such a procedure.

We introduce different models which arise naturally from the ways in which the discarded data can be treated, leading to truncated or censored likelihoods, as well as to a likelihood based on an only outliers gross errors model. Results on existence, uniqueness, robustness and asymptotic properties of the proposed estimators are included. A remarkable fact is that the proposed estimators generally keep the breakdown point of the initial (robust) estimators, but they could improve the rate of convergence of the initial estimator because our estimators always converge at rate n^1/2, independently of the rate of convergence of the initial estimator.

We consider the problem of computing an approximation to the integral I=∫_{[0, 1]^d} f(x) dx. Monte Carlo (MC) sampling typically attains a root mean squared error (RMSE) of O(n^−1/2) from n independent random function evaluations. By contrast, quasi-Monte Carlo (QMC) sampling using carefully equispaced evaluation points can attain the rate O(n^−1+ɛ) for any ɛ>0 and randomized QMC (RQMC) can attain the RMSE O(n^−3/2+ɛ), both under mild conditions on f.

Classical variance reduction methods for MC can be adapted to QMC. Published results combining QMC with importance sampling and with control variates have found worthwhile improvements, but no change in the error rate. This paper extends the classical variance reduction method of antithetic sampling and combines it with RQMC. One such method is shown to bring a modest improvement in the RMSE rate, attaining O(n^{−3/2−1/d+ɛ}) for any ɛ>0, for smooth enough f.

In the last decade, sequential Monte Carlo methods (SMC) emerged as a key tool in computational statistics [see, e.g., Sequential Monte Carlo Methods in Practice (2001) Springer, New York, Monte Carlo Strategies in Scientific Computing (2001) Springer, New York, Complex Stochastic Systems (2001) 109–173]. These algorithms approximate a sequence of distributions by a sequence of weighted empirical measures associated to a weighted population of particles, which are generated recursively.

Despite many theoretical advances [see, e.g., J. Roy. Statist. Soc. Ser. B 63 (2001) 127–146, Ann. Statist. 33 (2005) 1983–2021, Feynman–Kac Formulae. Genealogical and Interacting Particle Systems with Applications (2004) Springer, Ann. Statist. 32 (2004) 2385–2411], the large-sample theory of these approximations remains a question of central interest. In this paper we establish a law of large numbers and a central limit theorem as the number of particles gets large. We introduce the concepts of weighted sample consistency and asymptotic normality, and derive conditions under which the transformations of the weighted sample used in the SMC algorithm preserve these properties. To illustrate our findings, we analyze SMC algorithms to approximate the filtering distribution in state-space models. We show how our techniques allow to relax restrictive technical conditions used in previously reported works and provide grounds to analyze more sophisticated sequential sampling strategies, including branching, resampling at randomly selected times, and so on.

In this paper we provide a provably convergent algorithm for the multivariate Gaussian Maximum Likelihood version of the Behrens–Fisher Problem. Our work builds upon a formulation of the log-likelihood function proposed by Buot and Richards [5]. Instead of focusing on the first order optimality conditions, the algorithm aims directly for the maximization of the log-likelihood function itself to achieve a global solution. Convergence proof and complexity estimates are provided for the algorithm. Computational experiments illustrate the applicability of such methods to high-dimensional data. We also discuss how to extend the proposed methodology to a broader class of problems.

We establish a systematic algebraic relation between the Wald, Likelihood Ratio and Lagrangian Multiplier Test (W≥LR≥LM) in the context of the Behrens–Fisher Problem. Moreover, we use our algorithm to computationally investigate the finite-sample size and power of the Wald, Likelihood Ratio and Lagrange Multiplier Tests, which previously were only available through asymptotic results. The methods developed here are applicable to much higher dimensional settings than the ones available in the literature. This allows us to better capture the role of high dimensionality on the actual size and power of the tests for finite samples.

We show that the class of conditional distributions satisfying the coarsening at random (CAR) property for discrete data has a simple and robust algorithmic description based on randomized uniform multicovers: combinatorial objects generalizing the notion of partition of a set. However, the complexity of a given CAR mechanism can be large: the maximal “height” of the needed multicovers can be exponential in the number of points in the sample space. The results stem from a geometric interpretation of the set of CAR distributions as a convex polytope and a characterization of its extreme points. The hierarchy of CAR models defined in this way could be useful in parsimonious statistical modeling of CAR mechanisms, though the results also raise doubts in applied work as to the meaningfulness of the CAR assumption in its full generality.

We consider evaluation of proper posterior distributions obtained from improper prior distributions. Our context is estimating a bounded function φ of a parameter when the loss is quadratic. If the posterior mean of φ is admissible for all bounded φ, the posterior is strongly admissible. We give sufficient conditions for strong admissibility. These conditions involve the recurrence of a Markov chain associated with the estimation problem. We develop general sufficient conditions for recurrence of general state space Markov chains that are also of independent interest. Our main example concerns the p-dimensional multivariate normal distribution with mean vector θ when the prior distribution has the form g(‖θ‖²) dθ on the parameter space ℝ^p. Conditions on g for strong admissibility of the posterior are provided.

This paper studies the residual empirical process of long- and short-memory time series regression models and establishes its uniform expansion under a general framework. The results are applied to the stochastic regression models and unstable autoregressive models. For the long-memory noise, it is shown that the limit distribution of the Kolmogorov–Smirnov test statistic studied in Ho and Hsing [Ann. Statist. 24 (1996) 992–1024] does not hold when the stochastic regression model includes an unknown intercept or when the characteristic polynomial of the unstable autoregressive model has a unit root. To this end, two new statistics are proposed to test for the distribution of the long-memory noises of stochastic regression models and unstable autoregressive models.

Local increases in the mean of a random field are detected (conservatively) by thresholding a field of test statistics at a level u chosen to control the tail probability or p-value of its maximum. This p-value is approximated by the expected Euler characteristic (EC) of the excursion set of the test statistic field above u, denoted $\mathbb{E}\varphi(A_{u})$. Under isotropy, one can use the expansion $\mathbb{E}\varphi(A_{u})=\sum_{k}\mathcal{V}_{k}\rho_{k}(u)$, where $\mathcal{V}_{k}$ is an intrinsic volume of the parameter space and ρ_k is an EC density of the field. EC densities are available for a number of processes, mainly those constructed from (multivariate) Gaussian fields via smooth functions. Using saddlepoint methods, we derive an expansion for ρ_k(u) for fields which are only approximately Gaussian, but for which higher-order cumulants are available. We focus on linear combinations of n independent non-Gaussian fields, whence a Central Limit theorem is in force. The threshold u is allowed to grow with the sample size n, in which case our expression has a smaller relative asymptotic error than the Gaussian EC density. Several illustrative examples including an application to “bubbles” data accompany the theory.

Moving from univariate to bivariate jointly dependent long-memory time series introduces a phase parameter (γ), at the frequency of principal interest, zero; for short-memory series γ=0 automatically. The latter case has also been stressed under long memory, along with the “fractional differencing” case γ=(δ₂−δ₁)π/2, where δ₁, δ₂ are the memory parameters of the two series. We develop time domain conditions under which these are and are not relevant, and relate the consequent properties of cross-autocovariances to ones of the (possibly bilateral) moving average representation which, with martingale difference innovations of arbitrary dimension, is used in asymptotic theory for local Whittle parameter estimates depending on a single smoothing number. Incorporating also a regression parameter (β) which, when nonzero, indicates cointegration, the consistency proof of these implicitly defined estimates is nonstandard due to the β estimate converging faster than the others. We also establish joint asymptotic normality of the estimates, and indicate how this outcome can apply in statistical inference on several questions of interest. Issues of implemention are discussed, along with implications of knowing β and of correct or incorrect specification of γ, and possible extensions to higher-dimensional systems and nonstationary series.

We consider the nonparametric functional estimation of the drift of a Gaussian process via minimax and Bayes estimators. In this context, we construct superefficient estimators of Stein type for such drifts using the Malliavin integration by parts formula and superharmonic functionals on Gaussian space. Our results are illustrated by numerical simulations and extend the construction of James–Stein type estimators for Gaussian processes by Berger and Wolpert [J. Multivariate Anal. 13 (1983) 401–424].