We consider the nonparametric estimation of a periodic function that is observed in additive Gaussian white noise after convolution with a “boxcar,” the indicator function of an interval. This is an idealized model for the problem of recovery of noisy signals and images observed with “motion blur.” If the length of the boxcar is rational, then certain frequencies are irretreviably lost in the periodic model. We consider the rate of convergence of estimators when the length of the boxcar is irrational, using classical results on approximation of irrationals by continued fractions. A basic question of interest is whether the minimax rate of convergence is slower than for nonperiodic problems with 1/f-like convolution filters. The answer turns out to depend on the type and smoothness of functions being estimated in a manner not seen with “homogeneous” filters.

A nonparametric adaptation theory is developed for the construction of confidence intervals for linear functionals. A between class modulus of continuity captures the expected length of adaptive confidence intervals. Sharp lower bounds are given for the expected length and an ordered modulus of continuity is used to construct adaptive confidence procedures which are within a constant factor of the lower bounds. In addition, minimax theory over nonconvex parameter spaces is developed.

We consider the problem of constructing robust nonparametric confidence intervals and tests of hypothesis for the median when the data distribution is unknown and the data may contain a small fraction of contamination. We propose a modification of the sign test (and its associated confidence interval) which attains the nominal significance level (probability coverage) for any distribution in the contamination neighborhood of a continuous distribution. We also define some measures of robustness and efficiency under contamination for confidence intervals and tests. These measures are computed for the proposed procedures.

Generalized likelihood ratio statistics have been proposed in Fan, Zhang and Zhang [Ann. Statist. 29 (2001) 153–193] as a generally applicable method for testing nonparametric hypotheses about nonparametric functions. The likelihood ratio statistics are constructed based on the assumption that the distributions of stochastic errors are in a certain parametric family. We extend their work to the case where the error distribution is completely unspecified via newly proposed sieve empirical likelihood ratio (SELR) tests. The approach is also applied to test conditional estimating equations on the distributions of stochastic errors. It is shown that the proposed SELR statistics follow asymptotically rescaled χ²-distributions, with the scale constants and the degrees of freedom being independent of the nuisance parameters. This demonstrates that the Wilks phenomenon observed in Fan, Zhang and Zhang [Ann. Statist. 29 (2001) 153–193] continues to hold under more relaxed models and a larger class of techniques. The asymptotic power of the proposed test is also derived, which achieves the optimal rate for nonparametric hypothesis testing. The proposed approach has two advantages over the generalized likelihood ratio method: it requires one only to specify some conditional estimating equations rather than the entire distribution of the stochastic error, and the procedure adapts automatically to the unknown error distribution including heteroscedasticity. A simulation study is conducted to evaluate our proposed procedure empirically.

This article pursues a statistical study of the Hough transform, the celebrated computer vision algorithm used to detect the presence of lines in a noisy image. We first study asymptotic properties of the Hough transform estimator, whose objective is to find the line that “best” fits a set of planar points. In particular, we establish strong consistency and rates of convergence, and characterize the limiting distribution of the Hough transform estimator. While the convergence rates are seen to be slower than those found in some standard regression methods, the Hough transform estimator is shown to be more robust as measured by its breakdown point. We next study the Hough transform in the context of the problem of detecting multiple lines. This is addressed via the framework of excess mass functionals and modality testing. Throughout, several numerical examples help illustrate various properties of the estimator. Relations between the Hough transform and more mainstream statistical paradigms and methods are discussed as well.

We suggest an adaptive sampling rule for obtaining information from noisy signals using wavelet methods. The technique involves increasing the sampling rate when relatively high-frequency terms are incorporated into the wavelet estimator, and decreasing it when, again using thresholded terms as an empirical guide, signal complexity is judged to have decreased. Through sampling in this way the algorithm is able to accurately recover relatively complex signals without increasing the long-run average expense of sampling. It achieves this level of performance by exploiting the opportunities for near-real time sampling that are available if one uses a relatively high primary resolution level when constructing the basic wavelet estimator. In the practical problems that motivate the work, where signal to noise ratio is particularly high and the long-run average sampling rate may be several hundred thousand operations per second, high primary resolution levels are quite feasible.

In recent years a popular nonparametric model for coarsened data is an assumption on the coarsening mechanism called coarsening at random (CAR). It has been conjectured in several papers that this assumption cannot be tested by the data, that is, the assumption does not restrict the possible distributions of the data. In this paper we will show that this conjecture is not always true; an example will be current status data. We will also give conditions when the conjecture is true, and in doing so, we will introduce a generalized version of the CAR assumption. As an illustration, we retrieve the well-known result that the CAR assumption cannot be tested in the case of right-censored data.

We consider the problem of determining the optimal block (or subsample) size for a spatial subsampling method for spatial processes observed on regular grids. We derive expansions for the mean square error of the subsampling variance estimator, which yields an expression for the theoretically optimal block size. The optimal block size is shown to depend in an intricate way on the geometry of the spatial sampling region as well as characteristics of the underlying random field. Final expressions for the optimal block size make use of some nontrivial estimates of lattice point counts in shifts of convex sets. Optimal block sizes are computed for sampling regions of a number of commonly encountered shapes. Numerical studies are performed to compare subsampling methods as well as procedures for estimating the theoretically best block size.

We use martingales to study Bayesian consistency. We derive sufficient conditions for both Hellinger and Kullback–Leibler consistency, which do not rely on the use of a sieve. Alternative sufficient conditions for Hellinger consistency are also found and demonstrated on examples.

The posterior distribution of the number of components k in a finite mixture satisfies a set of inequality constraints. The result holds irrespective of the parametric form of the mixture components and under assumptions on the prior distribution weaker than those routinely made in the literature on Bayesian analysis of finite mixtures. The inequality constraints can be used to perform an “internal” consistency check of MCMC estimates of the posterior distribution of k and to provide improved estimates which are required to satisfy the constraints. Bounds on the posterior probability of k components are derived using the constraints. Implications on prior distribution specification and on the adequacy of the posterior distribution of k as a tool for selecting an adequate number of components in the mixture are also explored.

This paper establishes the global asymptotic equivalence between a Poisson process with variable intensity and white noise with drift under sharp smoothness conditions on the unknown function. This equivalence is also extended to density estimation models by Poissonization. The asymptotic equivalences are established by constructing explicit equivalence mappings. The impact of such asymptotic equivalence results is that an investigation in one of these nonparametric models automatically yields asymptotically analogous results in the other models.

We discuss properties of two methods for ascribing probabilities to the shape of a probability distribution. One is based on the idea of counting the number of modes of a bootstrap version of a standard kernel density estimator. We argue that the simplest form of that method suffers from the same difficulties that inhibit level accuracy of Silverman’s bandwidth-based test for modality: the conditional distribution of the bootstrap form of a density estimator is not a good approximation to the actual distribution of the estimator. This difficulty is less pronounced if the density estimator is oversmoothed, but the problem of selecting the extent of oversmoothing is inherently difficult. It is shown that the optimal bandwidth, in the sense of producing optimally high sensitivity, depends on the widths of putative bumps in the unknown density and is exactly as difficult to determine as those bumps are to detect. We also develop a second approach to ascribing a probability to shape, using Müller and Sawitzki’s notion of excess mass. In contrast to the context just discussed, it is shown that the bootstrap distribution of empirical excess mass is a relatively good approximation to its true distribution. This leads to empirical approximations to the likelihoods of different levels of “modal sharpness,” or “delineation,” of modes of a density. The technique is illustrated numerically.

It is well known that the number of modes of a kernel density estimator is monotone nonincreasing in the bandwidth if the kernel is a Gaussian density. There is numerical evidence of nonmonotonicity in the case of some non-Gaussian kernels, but little additional information is available. The present paper provides theoretical and numerical descriptions of the extent to which the number of modes is a nonmonotone function of bandwidth in the case of general compactly supported densities. Our results address popular kernels used in practice, for example, the Epanechnikov, biweight and triweight kernels, and show that in such cases nonmonotonicity is present with strictly positive probability for all sample sizes n≥3. In the Epanechnikov and biweight cases the probability of nonmonotonicity equals 1 for all n≥2. Nevertheless, in spite of the prevalence of lack of monotonicity revealed by these results, it is shown that the notion of a critical bandwidth (the smallest bandwidth above which the number of modes is guaranteed to be monotone) is still well defined. Moreover, just as in the Gaussian case, the critical bandwidth is of the same size as the bandwidth that minimises mean squared error of the density estimator. These theoretical results, and new numerical evidence, show that the main effects of nonmonotonicity occur for relatively small bandwidths, and have negligible impact on many aspects of bump hunting.

For a broad class of nonlinear regression models we investigate the local E- and c-optimal design problem. It is demonstrated that in many cases the optimal designs with respect to these optimality criteria are supported at the Chebyshev points, which are the local extrema of the equi-oscillating best approximation of the function f₀≡0 by a normalized linear combination of the regression functions in the corresponding linearized model. The class of models includes rational, logistic and exponential models and for the rational regression models the E- and c-optimal design problem is solved explicitly in many cases.

Factorial designs have broad applications in agricultural, engineering and scientific studies. In constructing and studying properties of factorial designs, traditional design theory treats all factors as nominal. However, this is not appropriate for experiments that involve quantitative factors. For designs with quantitative factors, level permutation of one or more factors in a design matrix could result in different geometric structures, and, thus, different design properties. In this paper indicator functions are introduced to represent factorial designs. A polynomial form of indicator functions is used to characterize the geometric structure of those designs. Geometric isomorphism is defined for classifying designs with quantitative factors. Based on indicator functions, a new aberration criteria is proposed and some minimum aberration designs are presented.

We provide a general method to analyze the asymptotic properties of a variety of estimators of continuous time diffusion processes when the data are not only discretely sampled in time but the time separating successive observations may possibly be random. We introduce a new operator, the generalized infinitesimal generator, to obtain Taylor expansions of the asymptotic moments of the estimators. As a special case, our results apply to the situation where the data are discretely sampled at a fixed nonrandom time interval. We include as specific examples estimators based on maximum-likelihood and discrete approximations such as the Euler scheme.

We study the problem of estimating the coefficients of a diffusion (X_t,t≥0); the estimation is based on discrete data X_nΔ,n=0,1,…,N. The sampling frequency Δ⁻¹ is constant, and asymptotics are taken as the number N of observations tends to infinity. We prove that the problem of estimating both the diffusion coefficient (the volatility) and the drift in a nonparametric setting is ill-posed: the minimax rates of convergence for Sobolev constraints and squared-error loss coincide with that of a, respectively, first- and second-order linear inverse problem. To ensure ergodicity and limit technical difficulties we restrict ourselves to scalar diffusions living on a compact interval with reflecting boundary conditions.

Our approach is based on the spectral analysis of the associated Markov semigroup. A rate-optimal estimation of the coefficients is obtained via the nonparametric estimation of an eigenvalue–eigenfunction pair of the transition operator of the discrete time Markov chain (X_nΔ,n=0,1,…,N) in a suitable Sobolev norm, together with an estimation of its invariant density.

An autoregressive process with Markov regime is an autoregressive process for which the regression function at each time point is given by a nonobservable Markov chain. In this paper we consider the asymptotic properties of the maximum likelihood estimator in a possibly nonstationary process of this kind for which the hidden state space is compact but not necessarily finite. Consistency and asymptotic normality are shown to follow from uniform exponential forgetting of the initial distribution for the hidden Markov chain conditional on the observations.

Let ξ₀,ξ₁,…,ξ_ω−1 be observations from the hidden Markov model with probability distribution P^θ₀, and let ξ_ω,ξ_ω+1,… be observations from the hidden Markov model with probability distribution P^θ₁. The parameters θ₀ and θ₁ are given, while the change point ω is unknown. The problem is to raise an alarm as soon as possible after the distribution changes from P^θ₀ to P^θ₁, but to avoid false alarms. Specifically, we seek a stopping rule N which allows us to observe the ξ's sequentially, such that E_∞N is large, and subject to this constraint, sup _kE_k(N−k|N≥k) is as small as possible. Here E_k denotes expectation under the change point k, and E_∞ denotes expectation under the hypothesis of no change whatever.

In this paper we investigate the performance of the Shiryayev–Roberts–Pollak (SRP) rule for change point detection in the dynamic system of hidden Markov models. By making use of Markov chain representation for the likelihood function, the structure of asymptotically minimax policy and of the Bayes rule, and sequential hypothesis testing theory for Markov random walks, we show that the SRP procedure is asymptotically minimax in the sense of Pollak [Ann. Statist. 13 (1985) 206–227]. Next, we present a second-order asymptotic approximation for the expected stopping time of such a stopping scheme when ω=1. Motivated by the sequential analysis in hidden Markov models, a nonlinear renewal theory for Markov random walks is also given.