We place ourselves in the setting of high-dimensional statistical inference where the number of variables p in a dataset of interest is of the same order of magnitude as the number of observations n.

We consider the spectrum of certain kernel random matrices, in particular n×n matrices whose (i, j)th entry is f(X'_iX_j/p) or f(‖X_i−X_j‖²/p) where p is the dimension of the data, and X_i are independent data vectors. Here f is assumed to be a locally smooth function.

The study is motivated by questions arising in statistics and computer science where these matrices are used to perform, among other things, nonlinear versions of principal component analysis. Surprisingly, we show that in high-dimensions, and for the models we analyze, the problem becomes essentially linear—which is at odds with heuristics sometimes used to justify the usage of these methods. The analysis also highlights certain peculiarities of models widely studied in random matrix theory and raises some questions about their relevance as tools to model high-dimensional data encountered in practice.

We study generalized bootstrap confidence regions for the mean of a random vector whose coordinates have an unknown dependency structure. The random vector is supposed to be either Gaussian or to have a symmetric and bounded distribution. The dimensionality of the vector can possibly be much larger than the number of observations and we focus on a nonasymptotic control of the confidence level, following ideas inspired by recent results in learning theory. We consider two approaches, the first based on a concentration principle (valid for a large class of resampling weights) and the second on a resampled quantile, specifically using Rademacher weights. Several intermediate results established in the approach based on concentration principles are of interest in their own right. We also discuss the question of accuracy when using Monte Carlo approximations of the resampled quantities.

In the context of correlated multiple tests, we aim to nonasymptotically control the family-wise error rate (FWER) using resampling-type procedures. We observe repeated realizations of a Gaussian random vector in possibly high dimension and with an unknown covariance matrix, and consider the one- and two-sided multiple testing problem for the mean values of its coordinates. We address this problem by using the confidence regions developed in the companion paper [Ann. Statist. (2009), to appear], which lead directly to single-step procedures; these can then be improved using step-down algorithms, following an established general methodology laid down by Romano and Wolf [J. Amer. Statist. Assoc. 100 (2005) 94–108]. This gives rise to several different procedures, whose performances are compared using simulated data.

An important estimation problem that is closely related to large-scale multiple testing is that of estimating the null density and the proportion of nonnull effects. A few estimators have been introduced in the literature; however, several important problems, including the evaluation of the minimax rate of convergence and the construction of rate-optimal estimators, remain open.

In this paper, we consider optimal estimation of the null density and the proportion of nonnull effects. Both minimax lower and upper bounds are derived. The lower bound is established by a two-point testing argument, where at the core is the novel construction of two least favorable marginal densities f₁ and f₂. The density f₁ is heavy tailed both in the spatial and frequency domains and f₂ is a perturbation of f₁ such that the characteristic functions associated with f₁ and f₂ match each other in low frequencies. The minimax upper bound is obtained by constructing estimators which rely on the empirical characteristic function and Fourier analysis. The estimator is shown to be minimax rate optimal.

Compared to existing methods in the literature, the proposed procedure not only provides more precise estimates of the null density and the proportion of the nonnull effects, but also yields more accurate results when used inside some multiple testing procedures which aim at controlling the False Discovery Rate (FDR). The procedure is easy to implement and numerical results are given.

In this paper, we investigate the asymptotic properties of nonparametric Bayesian mixtures of Betas for estimating a smooth density on [0, 1]. We consider a parametrization of Beta distributions in terms of mean and scale parameters and construct a mixture of these Betas in the mean parameter, while putting a prior on this scaling parameter. We prove that such Bayesian nonparametric models have good frequentist asymptotic properties. We determine the posterior rate of concentration around the true density and prove that it is the minimax rate of concentration when the true density belongs to a Hölder class with regularity β, for all positive β, leading to a minimax adaptive estimating procedure of the density. We also believe that the approximating results obtained on these mixtures of Beta densities can be of interest in a frequentist framework.

We consider the statistical experiment given by a sample y(1), …, y(n) of a stationary Gaussian process with an unknown smooth spectral density f. Asymptotic equivalence, in the sense of Le Cam’s deficiency Δ-distance, to two Gaussian experiments with simpler structure is established. The first one is given by independent zero mean Gaussians with variance approximately f(ω_i), where ω_i is a uniform grid of points in (−π, π) (nonparametric Gaussian scale regression). This approximation is closely related to well-known asymptotic independence results for the periodogram and corresponding inference methods. The second asymptotic equivalence is to a Gaussian white noise model where the drift function is the log-spectral density. This represents the step from a Gaussian scale model to a location model, and also has a counterpart in established inference methods, that is, log-periodogram regression. The problem of simple explicit equivalence maps (Markov kernels), allowing to directly carry over inference, appears in this context but is not solved here.

This paper studies an approximation method for the log-likelihood function of a nonlinear diffusion process using the bridge of the diffusion. The main result (Theorem 1) shows that this approximation converges uniformly to the unknown likelihood function and can therefore be used efficiently with any algorithm for sampling from the law of the bridge. We also introduce an expected maximum likelihood (EML) algorithm for inferring the parameters of discretely observed diffusion processes. The approach is applicable to a subclass of nonlinear SDEs with constant volatility and drift that is linear in the model parameters. In this setting, globally optimal parameters are obtained in a single step by solving a linear system. Simulation studies to test the EML algorithm show that it performs well when compared with algorithms based on the exact maximum likelihood as well as closed-form likelihood expansions.

In this paper, we study the estimation for a partial-linear single-index model. A two-stage estimation procedure is proposed to estimate the link function for the single index and the parameters in the single index, as well as the parameters in the linear component of the model. Asymptotic normality is established for both parametric components. For the index, a constrained estimating equation leads to an asymptotically more efficient estimator than existing estimators in the sense that it is of a smaller limiting variance. The estimator of the nonparametric link function achieves optimal convergence rates, and the structural error variance is obtained. In addition, the results facilitate the construction of confidence regions and hypothesis testing for the unknown parameters. A simulation study is performed and an application to a real dataset is illustrated. The extension to multiple indices is briefly sketched.

This paper presents a study of the large-sample behavior of the posterior distribution of a structural parameter which is partially identified by moment inequalities. The posterior density is derived based on the limited information likelihood. The posterior distribution converges to zero exponentially fast on any δ-contraction outside the identified region. Inside, it is bounded below by a positive constant if the identified region is assumed to have a nonempty interior. Our simulation evidence indicates that the Bayesian approach has advantages over frequentist methods, in the sense that, with a proper choice of the prior, the posterior provides more information about the true parameter inside the identified region. We also address the problem of moment and model selection. Our optimality criterion is the maximum posterior procedure and we show that, asymptotically, it selects the true moment/model combination with the most moment inequalities and the simplest model.

We consider the problem of estimating the fractional order of a Lévy process from low frequency historical and options data. An estimation methodology is developed which allows us to treat both estimation and calibration problems in a unified way. The corresponding procedure consists of two steps: the estimation of a conditional characteristic function and the weighted least squares estimation of the fractional order in spectral domain. While the second step is identical for both calibration and estimation, the first one depends on the problem at hand. Minimax rates of convergence for the fractional order estimate are derived, the asymptotic normality is proved and a data-driven algorithm based on aggregation is proposed. The performance of the estimator in both estimation and calibration setups is illustrated by a simulation study.

We consider the problem of estimating the distribution function, the density and the hazard rate of the (unobservable) event time in the current status model. A well studied and natural nonparametric estimator for the distribution function in this model is the nonparametric maximum likelihood estimator (MLE). We study two alternative methods for the estimation of the distribution function, assuming some smoothness of the event time distribution. The first estimator is based on a maximum smoothed likelihood approach. The second method is based on smoothing the (discrete) MLE of the distribution function. These estimators can be used to estimate the density and hazard rate of the event time distribution based on the plug-in principle.

In this paper, we derive valid Edgeworth expansions for studentized versions of a large class of statistics when the data are generated by a strongly mixing process. Under dependence, the asymptotic variance of such a statistic is given by an infinite series of lag-covariances, and therefore, studentizing factors (i.e., estimators of the asymptotic standard error) typically involve an increasing number, say, ℓ of lag-covariance estimators, which are themselves quadratic functions of the observations. The unboundedness of the dimension ℓ of these quadratic functions makes the derivation and the form of the expansions nonstandard. It is shown that in contrast to the case of the studentized means under independence, the derived Edgeworth expansion is a superposition of three distinct series, respectively, given by one in powers of n^−1/2, one in powers of [n/ℓ]^−1/2 (resulting from the standard error of the studentizing factor) and one in powers of the bias of the studentizing factor, where n denotes the sample size.

Ordinary differential equations (ODEs) are commonly used to model dynamic behavior of a system. Because many parameters are unknown and have to be estimated from the observed data, there is growing interest in statistics to develop efficient estimation procedures for these parameters. Among the proposed methods in the literature, the generalized profiling estimation method developed by Ramsay and colleagues is particularly promising for its computational efficiency and good performance. In this approach, the ODE solution is approximated with a linear combination of basis functions. The coefficients of the basis functions are estimated by a penalized smoothing procedure with an ODE-defined penalty. However, the statistical properties of this procedure are not known. In this paper, we first give an upper bound on the uniform norm of the difference between the true solutions and their approximations. Then we use this bound to prove the consistency and asymptotic normality of this estimation procedure. We show that the asymptotic covariance matrix is the same as that of the maximum likelihood estimation. Therefore, this procedure is asymptotically efficient. For a fixed sample and fixed basis functions, we study the limiting behavior of the approximation when the smoothing parameter tends to infinity. We propose an algorithm to choose the smoothing parameters and a method to compute the deviation of the spline approximation from solution without solving the ODEs.

We study estimation of a multivariate function f : R^d→R when the observations are available from the function Af, where A is a known linear operator. Both the Gaussian white noise model and density estimation are studied. We define an L₂-empirical risk functional which is used to define a δ-net minimizer and a dense empirical risk minimizer. Upper bounds for the mean integrated squared error of the estimators are given. The upper bounds show how the difficulty of the estimation depends on the operator through the norm of the adjoint of the inverse of the operator and on the underlying function class through the entropy of the class. Corresponding lower bounds are also derived. As examples, we consider convolution operators and the Radon transform. In these examples, the estimators achieve the optimal rates of convergence. Furthermore, a new type of oracle inequality is given for inverse problems in additive models.

P-values have been the focus of considerable criticism based on various considerations. Still, the P-value represents one of the most commonly used statistical tools. When assessing the suitability of a single hypothesized distribution, it is not clear that there is a better choice for a measure of surprise. This paper is concerned with the definition of appropriate model-based P-values for model checking.

Under mild assumptions on the kernel, we obtain the best known error rates in a regularized learning scenario taking place in the corresponding reproducing kernel Hilbert space (RKHS). The main novelty in the analysis is a proof that one can use a regularization term that grows significantly slower than the standard quadratic growth in the RKHS norm.

Let {Z_n} be a real nonstationary stochastic process such that $E(Z_{n}|{\mathcal{F}}_{n-1})\stackrel{\mathrm{a.s.}}<\infty$ and $E(Z_{n}^{2}|{\mathcal{F}}_{n-1})\stackrel{\mathrm{a.s.}}<\infty$, where $\{{\mathcal{F}}_{n}\}$ is an increasing sequence of σ-algebras. Assuming that $E(Z_{n}|{\mathcal{F}}_{n-1})=g_{n}(\theta_{0},\nu_{0})=g_{n}^{(1)}(\theta_{0})+g_{n}^{(2)}(\theta _{0},\nu_{0})$, θ₀∈ℝ^p, p<∞, ν₀∈ℝ^q and q≤∞, we study the asymptotic properties of $\widehat{\theta}_{n}:=\arg\min_{\theta}\sum_{k=1}^{n}(Z_{k}-g_{k}({\theta,\widehat{\nu}}))^{2}\lambda _{k}^{-1}$, where λ_k is ${\mathcal{F}}_{k-1}$-measurable, ̂ν={̂ν_k} is a sequence of estimations of ν₀, g_n(θ, ̂ν) is Lipschitz in θ and g_n⁽²⁾(θ₀, ̂ν)−g_n⁽²⁾(θ, ̂ν) is asymptotically negligible relative to g_n⁽¹⁾(θ₀)−g_n⁽¹⁾(θ). We first generalize to this nonlinear stochastic model the necessary and sufficient condition obtained for the strong consistency of {̂θ_n} in the linear model. For that, we prove a strong law of large numbers for a class of submartingales. Again using this strong law, we derive the general conditions leading to the asymptotic distribution of ̂θ_n. We illustrate the theoretical results with examples of branching processes, and extension to quasi-likelihood estimators is also considered.

Consider the problem of testing s hypotheses simultaneously. In this paper, we derive methods which control the generalized family-wise error rate given by the probability of k or more false rejections, abbreviated k-FWER. We derive both single-step and step-down procedures that control the k-FWER in finite samples or asymptotically, depending on the situation. Moreover, the procedures are asymptotically balanced in an appropriate sense. We briefly consider control of the average number of false rejections. Additionally, we consider the false discovery proportion (FDP), defined as the number of false rejections divided by the total number of rejections (and defined to be 0 if there are no rejections). Here, the goal is to construct methods which satisfy, for given γ and α, P{FDP>γ}≤α, at least asymptotically. Special attention is paid to the construction of methods which implicitly take into account the dependence structure of the individual test statistics in order to further increase the ability to detect false null hypotheses. A general resampling and subsampling approach is presented which achieves these objectives, at least asymptotically.