In this paper we offer a unified approach to the problem of nonparametric regression on the unit interval. It is based on a universal, honest and nonasymptotic confidence region $\mathcal{A}_{n}$ which is defined by a set of linear inequalities involving the values of the functions at the design points. Interest will typically center on certain simplest functions in $\mathcal{A}_{n}$ where simplicity can be defined in terms of shape (number of local extremes, intervals of convexity/concavity) or smoothness (bounds on derivatives) or a combination of both. Once some form of regularization has been decided upon the confidence region can be used to provide honest nonasymptotic confidence bounds which are less informative but conceptually much simpler.

Stochastic approximation Monte Carlo (SAMC) has recently been proposed by Liang, Liu and Carroll [J. Amer. Statist. Assoc. 102 (2007) 305–320] as a general simulation and optimization algorithm. In this paper, we propose to improve its convergence using smoothing methods and discuss the application of the new algorithm to Bayesian model selection problems. The new algorithm is tested through a change-point identification example. The numerical results indicate that the new algorithm can outperform SAMC and reversible jump MCMC significantly for the model selection problems. The new algorithm represents a general form of the stochastic approximation Markov chain Monte Carlo algorithm. It allows multiple samples to be generated at each iteration, and a bias term to be included in the parameter updating step. A rigorous proof for the convergence of the general algorithm is established under verifiable conditions. This paper also provides a framework on how to improve efficiency of Monte Carlo simulations by incorporating some nonparametric techniques.

We consider nonparametric Bayesian estimation inference using a rescaled smooth Gaussian field as a prior for a multidimensional function. The rescaling is achieved using a Gamma variable and the procedure can be viewed as choosing an inverse Gamma bandwidth. The procedure is studied from a frequentist perspective in three statistical settings involving replicated observations (density estimation, regression and classification). We prove that the resulting posterior distribution shrinks to the distribution that generates the data at a speed which is minimax-optimal up to a logarithmic factor, whatever the regularity level of the data-generating distribution. Thus the hierachical Bayesian procedure, with a fixed prior, is shown to be fully adaptive.

We consider the limit distribution of maxima of periodograms for stationary processes. Our method is based on m-dependent approximation for stationary processes and a moderate deviation result.

We consider estimation of quantile curves for a general class of nonstationary processes. Consistency and central limit results are obtained for local linear quantile estimates under a mild short-range dependence condition. Our results are applied to environmental data sets. In particular, our results can be used to address the problem of whether climate variability has changed, an important problem raised by IPCC (Intergovernmental Panel on Climate Change) in 2001.

Strong consistency and asymptotic normality of the quasi-maximum likelihood estimator are given for a general class of multidimensional causal processes. For particular cases already studied in the literature [for instance univariate or multivariate ARCH(∞) processes], the assumptions required for establishing these results are often weaker than existing conditions. The QMLE asymptotic behavior is also given for numerous new examples of univariate or multivariate processes (for instance TARCH or NLARCH processes).

Consider the problem of estimating the γ-level set G_γ^*={x: f(x)≥γ} of an unknown d-dimensional density function f based on n independent observations X₁, …, X_n from the density. This problem has been addressed under global error criteria related to the symmetric set difference. However, in certain applications a spatially uniform mode of convergence is desirable to ensure that the estimated set is close to the target set everywhere. The Hausdorff error criterion provides this degree of uniformity and, hence, is more appropriate in such situations. It is known that the minimax optimal rate of error convergence for the Hausdorff metric is (n/log n)^−1/(d+2α) for level sets with boundaries that have a Lipschitz functional form, where the parameter α characterizes the regularity of the density around the level of interest. However, the estimators proposed in previous work are nonadaptive to the density regularity and require knowledge of the parameter α. Furthermore, previously developed estimators achieve the minimax optimal rate for rather restricted classes of sets (e.g., the boundary fragment and star-shaped sets) that effectively reduce the set estimation problem to a function estimation problem. This characterization precludes level sets with multiple connected components, which are fundamental to many applications. This paper presents a fully data-driven procedure that is adaptive to unknown regularity conditions and achieves near minimax optimal Hausdorff error control for a class of density level sets with very general shapes and multiple connected components.

This paper discusses the problem of adaptive estimation of a univariate object like the value of a regression function at a given point or a linear functional in a linear inverse problem. We consider an adaptive procedure originated from Lepski [Theory Probab. Appl. 35 (1990) 454–466.] that selects in a data-driven way one estimate out of a given class of estimates ordered by their variability. A serious problem with using this and similar procedures is the choice of some tuning parameters like thresholds. Numerical results show that the theoretically recommended proposals appear to be too conservative and lead to a strong oversmoothing effect. A careful choice of the parameters of the procedure is extremely important for getting the reasonable quality of estimation. The main contribution of this paper is the new approach for choosing the parameters of the procedure by providing the prescribed behavior of the resulting estimate in the simple parametric situation. We establish a non-asymptotical “oracle” bound, which shows that the estimation risk is, up to a logarithmic multiplier, equal to the risk of the “oracle” estimate that is optimally selected from the given family. A numerical study demonstrates a good performance of the resulting procedure in a number of simulated examples.

Ancestral graphs can encode conditional independence relations that arise in directed acyclic graph (DAG) models with latent and selection variables. However, for any ancestral graph, there may be several other graphs to which it is Markov equivalent. We state and prove conditions under which two maximal ancestral graphs are Markov equivalent to each other, thereby extending analogous results for DAGs given by other authors. These conditions lead to an algorithm for determining Markov equivalence that runs in time that is polynomial in the number of vertices in the graph.

We give a finite-sample analysis of predictive inference procedures after model selection in regression with random design. The analysis is focused on a statistically challenging scenario where the number of potentially important explanatory variables can be infinite, where no regularity conditions are imposed on unknown parameters, where the number of explanatory variables in a “good” model can be of the same order as sample size and where the number of candidate models can be of larger order than sample size. The performance of inference procedures is evaluated conditional on the training sample. Under weak conditions on only the number of candidate models and on their complexity, and uniformly over all data-generating processes under consideration, we show that a certain prediction interval is approximately valid and short with high probability in finite samples, in the sense that its actual coverage probability is close to the nominal one and in the sense that its length is close to the length of an infeasible interval that is constructed by actually knowing the “best” candidate model. Similar results are shown to hold for predictive inference procedures other than prediction intervals like, for example, tests of whether a future response will lie above or below a given threshold.

Principal component analysis (PCA) is a classical method for dimensionality reduction based on extracting the dominant eigenvectors of the sample covariance matrix. However, PCA is well known to behave poorly in the “large p, small n” setting, in which the problem dimension p is comparable to or larger than the sample size n. This paper studies PCA in this high-dimensional regime, but under the additional assumption that the maximal eigenvector is sparse, say, with at most k nonzero components. We consider a spiked covariance model in which a base matrix is perturbed by adding a k-sparse maximal eigenvector, and we analyze two computationally tractable methods for recovering the support set of this maximal eigenvector, as follows: (a) a simple diagonal thresholding method, which transitions from success to failure as a function of the rescaled sample size θ_dia(n, p, k)=n/[k²log(p−k)]; and (b) a more sophisticated semidefinite programming (SDP) relaxation, which succeeds once the rescaled sample size θ_sdp(n, p, k)=n/[klog(p−k)] is larger than a critical threshold. In addition, we prove that no method, including the best method which has exponential-time complexity, can succeed in recovering the support if the order parameter θ_sdp(n, p, k) is below a threshold. Our results thus highlight an interesting trade-off between computational and statistical efficiency in high-dimensional inference.

We consider estimating an unknown signal, both blocky and sparse, which is corrupted by additive noise. We study three interrelated least squares procedures and their asymptotic properties. The first procedure is the fused lasso, put forward by Friedman et al. [Ann. Appl. Statist. 1 (2007) 302–332], which we modify into a different estimator, called the fused adaptive lasso, with better properties. The other two estimators we discuss solve least squares problems on sieves; one constrains the maximal ℓ₁ norm and the maximal total variation seminorm, and the other restricts the number of blocks and the number of nonzero coordinates of the signal. We derive conditions for the recovery of the true block partition and the true sparsity patterns by the fused lasso and the fused adaptive lasso, and we derive convergence rates for the sieve estimators, explicitly in terms of the constraining parameters.

Consider a random sample from a bivariate distribution function F in the max-domain of attraction of an extreme-value distribution function G. This G is characterized by two extreme-value indices and a spectral measure, the latter determining the tail dependence structure of F. A major issue in multivariate extreme-value theory is the estimation of the spectral measure Φ_p with respect to the L_p norm. For every p∈[1, ∞], a nonparametric maximum empirical likelihood estimator is proposed for Φ_p. The main novelty is that these estimators are guaranteed to satisfy the moment constraints by which spectral measures are characterized. Asymptotic normality of the estimators is proved under conditions that allow for tail independence. Moreover, the conditions are easily verifiable as we demonstrate through a number of theoretical examples. A simulation study shows a substantially improved performance of the new estimators. Two case studies illustrate how to implement the methods in practice.

Consider a continuous random pair (X, Y) whose dependence is characterized by an extreme-value copula with Pickands dependence function A. When the marginal distributions of X and Y are known, several consistent estimators of A are available. Most of them are variants of the estimators due to Pickands [Bull. Inst. Internat. Statist. 49 (1981) 859–878] and Capéraà, Fougères and Genest [Biometrika 84 (1997) 567–577]. In this paper, rank-based versions of these estimators are proposed for the more common case where the margins of X and Y are unknown. Results on the limit behavior of a class of weighted bivariate empirical processes are used to show the consistency and asymptotic normality of these rank-based estimators. Their finite- and large-sample performance is then compared to that of their known-margin analogues, as well as with endpoint-corrected versions thereof. Explicit formulas and consistent estimates for their asymptotic variances are also given.

We reconsider the existing kernel estimators for a copula function, as proposed in Gijbels and Mielniczuk [Comm. Statist. Theory Methods 19 (1990) 445–464], Fermanian, Radulovič and Wegkamp [Bernoulli 10 (2004) 847–860] and Chen and Huang [Canad. J. Statist. 35 (2007) 265–282]. All of these estimators have as a drawback that they can suffer from a corner bias problem. A way to deal with this is to impose rather stringent conditions on the copula, outruling as such many classical families of copulas. In this paper, we propose improved estimators that take care of the typical corner bias problem. For Gijbels and Mielniczuk [Comm. Statist. Theory Methods 19 (1990) 445–464] and Chen and Huang [Canad. J. Statist. 35 (2007) 265–282], the improvement involves shrinking the bandwidth with an appropriate functional factor; for Fermanian, Radulovič and Wegkamp [Bernoulli 10 (2004) 847–860], this is done by using a transformation. The theoretical contribution of the paper is a weak convergence result for the three improved estimators under conditions that are met for most copula families. We also discuss the choice of bandwidth parameters, theoretically and practically, and illustrate the finite-sample behaviour of the estimators in a simulation study. The improved estimators are applied to goodness-of-fit testing for copulas.

It is shown that a necessary and sufficient condition for an Archimedean copula generator to generate a d-dimensional copula is that the generator is a d-monotone function. The class of d-dimensional Archimedean copulas is shown to coincide with the class of survival copulas of d-dimensional ℓ₁-norm symmetric distributions that place no point mass at the origin. The d-monotone Archimedean copula generators may be characterized using a little-known integral transform of Williamson [Duke Math. J. 23 (1956) 189–207] in an analogous manner to the well-known Bernstein–Widder characterization of completely monotone generators in terms of the Laplace transform. These insights allow the construction of new Archimedean copula families and provide a general solution to the problem of sampling multivariate Archimedean copulas. They also yield useful expressions for the d-dimensional Kendall function and Kendall’s rank correlation coefficients and facilitate the derivation of results on the existence of densities and the description of singular components for Archimedean copulas. The existence of a sharp lower bound for Archimedean copulas with respect to the positive lower orthant dependence ordering is shown.