This paper performs an asymptotic analysis of penalized spline estimators. We compare P-splines and splines with a penalty of the type used with smoothing splines. The asymptotic rates of the supremum norm of the difference between these two estimators over compact subsets of the interior and over the entire interval are established. It is shown that a P-spline and a smoothing spline are asymptotically equivalent provided that the number of knots of the P-spline is large enough, and the two estimators have the same equivalent kernels for both interior points and boundary points.

In Gaussian graphical models, the conjugate prior for the precision matrix K is called G-Wishart distribution, W_G(δ,D). In this paper we propose a new sampling method for the W_G(δ,D) based on the Metropolis Hastings algorithm and we show its validity through a number of numerical experiments. We show that this method can be easily used to estimate the Deviance Information Criterion, providing with a computationally inexpensive approach for model selection.

Let (X₁,Y₁),…,(X_n,Y_n) be independent and identically distributed random elements taking values in ℱ×ℋ, where ℱ is a semi-metric space and ℋ is a separable Hilbert space. We investigate the rates of strong (almost sure) convergence of the k-nearest neighbor estimate. We give two convergence results assuming a finite moment condition and exponential tail condition on the noises respectively, with the latter requiring less stringent conditions on k for convergence.

In this paper the behaviour of linear resampling statistics in martingale difference arrays X_n,i,i≤k(n) is studied. It is shown that different bootstrap and permutation procedures work if the array (X_n,i)_i fulfils the conditions of a general central limit theorem. As an application we obtain amongst others resampling versions of the Kuan and Lee [20] test for the martingale difference hypothesis.

We consider the problem of estimating the unknown response function in the multichannel deconvolution model with a boxcar-like kernel which is of particular interest in signal processing. It is known that, when the number of channels is finite, the precision of reconstruction of the response function increases as the number of channels M grow (even when the total number of observations n for all channels M remains constant) and this requires that the parameter of the channels form a Badly Approximable M-tuple.

Recent advances in data collection and recording techniques made it of urgent interest to study the case when the number of channels M=M_n grow with the total number of observations n. However, in real-life situations, the number of channels M=M_n usually refers to the number of physical devices and, consequently, may grow to infinity only at a slow rate as n→∞. Unfortunately, existing theoretical results cannot be blindly applied to accommodate the case when M=M_n→∞ as n→∞. This is due to the fact that, to the best of our knowledge, so far no one have studied the construction of a Badly Approximable M-tuple of a growing length on a specified interval, of a non-asymptotic length, of the real line, as M is growing. Therefore, this generalization requires non-trivial results in number theory.

When M=M_n grows slowly as n increases, we develop a procedure for the construction of a Badly Approximable M-tuple on a specified interval, of a non-asymptotic length, together with a lower bound associated with this M-tuple, which explicitly shows its dependence on M as M is growing. This result is further used for the evaluation of the L²-risk of the suggested adaptive wavelet thresholding estimator of the unknown response function and, furthermore, for the choice of the optimal number of channels M which minimizes the L²-risk.

Given a number of different studies estimating the same effect size, it is often desired to explain heterogeneity of outcomes using concomitant covariates. For very large sample sizes, effect size estimates are approximately normally distributed and a straightforward application of weighted least squares is appropriate. However in practice within study sample variances are often small to moderate, casting doubt on the normality assumption for effect sizes and results based on weighted least squares. One can alternatively variance stabilize the effect size estimates and adopt a generalized linear model. Both methods are compared on two examples when effect sizes are the standardized difference of means. Then simulation studies are conducted to compare the coverage and width of confidence intervals for the meta-regression coefficients. These simulations show that the coverage probability associated with weighted least squares can be well below the nominated level for small to moderate sample sizes. Further empirical investigations reveal a bias in estimation due to using estimated weights which were assumed to be known. For these models, the generalized linear model approach resulted in much improved coverage probabilities.

The problem of estimating the center of symmetry of a symmetric signal in Gaussian white noise is considered. The underlying nuisance function f is not assumed to be differentiable, which makes a new point of view to the problem necessary.

We investigate the well-known sieve maximum likelihood estimators based on the cumulated periodogram, and study minimax rates over classes of irregular functions. It is shown that if the class appropriately controls the growth to infinity of the Fisher information over the sieve, semiparametric fast rates of convergence are obtained. We prove a lower bound result which implies that these semiparametric rates are really slower than the parametric ones, contrary to the regular case. Our results also suggest that there may be room to improve on the popular cumulated periodogram estimator.

We consider the sparse regression model where the number of parameters p is larger than the sample size n. The difficulty when considering high-dimensional problems is to propose estimators achieving a good compromise between statistical and computational performances. The Lasso is solution of a convex minimization problem, hence computable for large value of p. However stringent conditions on the design are required to establish fast rates of convergence for this estimator. Dalalyan and Tsybakov [17–19] proposed an exponential weights procedure achieving a good compromise between the statistical and computational aspects. This estimator can be computed for reasonably large p and satisfies a sparsity oracle inequality in expectation for the empirical excess risk only under mild assumptions on the design. In this paper, we propose an exponential weights estimator similar to that of [17] but with improved statistical performances. Our main result is a sparsity oracle inequality in probability for the true excess risk.

We derive the first and the second moments of the Moore-Penrose generalized inverse of a singular standard Wishart matrix without relying on a density. Instead, we use the moments of an inverse Wishart distribution and an invariance argument which is related to the literature on tensor functions. We also find the order of the spectral norm of the generalized inverse of a Wishart matrix as its dimension and degrees of freedom diverge.

We consider kernel regression estimate when both the response variable and the explanatory one are functional. The rates of uniform almost complete convergence are stated as function of the small ball probability of the predictor and as function of the entropy of the set on which uniformity is obtained.

We consider the problem of predictive density estimation for normal models under Kullback-Leibler loss (KL loss) when the parameter space is constrained to a convex set. More particularly, we assume that $X\sim {\cal N}_{p}(\mu,v_{x}I)$ is observed and that we wish to estimate the density of $Y\sim {\cal N}_{p}(\mu,v_{y}I)$ under KL loss when μ is restricted to the convex set C⊂ℝ^p. We show that the best unrestricted invariant predictive density estimator p̂_U is dominated by the Bayes estimator p̂_πC associated to the uniform prior π_C on C. We also study so called plug-in estimators, giving conditions under which domination of one estimator of the mean vector μ over another under the usual quadratic loss, translates into a domination result for certain corresponding plug-in density estimators under KL loss. Risk comparisons and domination results are also made for comparisons of plug-in estimators and Bayes predictive density estimators. Additionally, minimaxity and domination results are given for the cases where: (i) C is a cone, and (ii) C is a ball.

We derive an upper bound for the mean of the supremum of the empirical process indexed by a class of functions that are known to have variance bounded by a small constant δ. The bound is expressed in the uniform entropy integral of the class at δ. The bound yields a rate of convergence of minimum contrast estimators when applied to the modulus of continuity of the contrast functions.

Motivated by a broad range of potential applications in topological and geometric inference, we introduce a weighted version of the k-nearest neighbor density estimate. Various pointwise consistency results of this estimate are established. We present a general central limit theorem under the lightest possible conditions. In addition, a strong approximation result is obtained and the choice of the optimal set of weights is discussed. In particular, the classical k-nearest neighbor estimate is not optimal in a sense described in the manuscript. The proposed method has been implemented to recover level sets in both simulated and real-life data.

Gaussian random fields are commonly used as models for spatial processes and maximum likelihood is a preferred method of choice for estimating the covariance parameters. However if the sample size n is large, evaluating the likelihood can be a numerical challenge. Covariance tapering is a way of approximating the covariance function with a taper (usually a compactly supported function) so that the computational burden is reduced. This article studies the fixed-domain asymptotic behavior of the tapered MLE for the microergodic parameter of a Matérn covariance function when the taper support is allowed to shrink as n→∞. In particular if the dimension of the underlying space is ≤3, conditions are established in which the tapered MLE is strongly consistent and also asymptotically normal. Numerical experiments are reported that gauge the quality of these approximations for finite n.

A Dirichlet mixture of exponential power distributions, as a prior on densities supported on the real line in the problem of Bayesian density estimation, is a natural generalization of a Dirichlet mixture of normals, which has been shown to possess good frequentist asymptotic properties in terms of posterior consistency and rates of convergence. In this article, we establish upper bounds on the rates of convergence for the posterior distribution of a Dirichlet mixture of exponential power densities, assuming that the true density has the same form as the model. When the kernel is analytic and the mixing distribution has either compact support or sub-exponential tails, a nearly parametric rate, up to a logarithmic factor whose exponent depends on the tail behaviour of the base measure of the Dirichlet process and the exponential decay rate at zero of the prior for the scale parameter, is obtained. The result covers the important special case where the true density is a location mixture of normals and shows that a nearly parametric rate arises also when the prior on the scale contains zero in its support, provided it has a sufficiently fast decay rate at zero. This improves on some recent results on density estimation with Dirichlet mixtures of normals by allowing the inverse-gamma distribution, which is a commonly used prior on the square of the bandwidth. When the kernel is not infinitely differentiable at zero, as the case may be depending on the shape parameter, the posterior distribution is shown to concentrate around the sampling density at a slower rate.

Use of redundant statistical model is often the case with practical data analysis. Redundancy widely investigated is inclusion of irrelevant predictors which is resolved by setting their coefficients to zero. On the other hand, it is also useful to consider overlapping parameters of which the values are similar. Grouping by regarding a set of parameters as a single parameter contributes to building intimate parameterization and increasing estimation accuracy by dimension reduction.

The paper proposes a data adaptive automatic grouping of parameters, which simultaneously enables variable selection that can yield sparse solution, by applying the smooth-thresholding. The new procedure is applicable to several estimation equation-based methods, and is shown to possess the oracle property. No convex optimization is needed for its implementation. Numerical examinations including large p small n situation are performed. Proposed automatic grouping applies to interaction modeling for Ohio wheeze data and for credit scoring data.

In many multiple testing procedures, accurate modeling of the p-value distribution is a key issue. Mixture distributions have been shown to provide adequate models for p-value densities under the null and the alternative hypotheses. An important parameter of the mixture model that needs to be estimated is the proportion of true null hypotheses, which under the mixture formulation becomes the probability mass attached to the value associated with the null hypothesis. It is well known that in a general mixture model, especially when a scale parameter is present, the mixing distribution need not be identifiable. Nevertheless, under our setting for mixture model for p-values, we show that the weight attached to the null hypothesis is identifiable under two very different types of conditions. We consider several examples including univariate and multivariate mixture models for transformed p-values. Finally, we formulate an abstract theorem for general mixtures and present other examples.

This paper first proves that the sample based Pearson’s product-moment correlation coefficient and the quotient correlation coefficient are asymptotically independent, which is a very important property as it shows that these two correlation coefficients measure completely different dependencies between two random variables, and they can be very useful if they are simultaneously applied to data analysis. Motivated from this fact, the paper introduces a new way of combining these two sample based correlation coefficients into maximal strength measures of variable association. Second, the paper introduces a new marginal distribution transformation method which is based on a rank-preserving scale regeneration procedure, and is distribution free. In testing hypothesis of independence between two continuous random variables, the limiting distributions of the combined measures are shown to follow a max-linear of two independent χ² random variables. The new measures as test statistics are compared with several existing tests. Theoretical results and simulation examples show that the new tests are clearly superior. In real data analysis, the paper proposes to incorporate nonlinear data transformation into the rank-preserving scale regeneration procedure, and a conditional expectation test procedure whose test statistic is shown to have a non-standard limit distribution. Data analysis results suggest that this new testing procedure can detect inherent dependencies in the data and could lead to a more meaningful decision making.

We study the Bernstein-von Mises (BvM) phenomenon, i.e., Bayesian credible sets and frequentist confidence regions for the estimation error coincide asymptotically, for the infinite-dimensional Gaussian white noise model governed by Gaussian prior with diagonal-covariance structure. While in parametric statistics this fact is a consequence of (a particular form of) the BvM Theorem, in the nonparametric setup, however, the BvM Theorem is known to fail even in some, apparently, elementary cases. In the present paper we show that BvM-like statements hold for this model, provided that the parameter space is suitably embedded into the support of the prior. The overall conclusion is that, unlike in the parametric setup, positive results regarding frequentist probability coverage of credible sets can only be obtained if the prior assigns null mass to the parameter space.

This study introduces a new method of visualizing complex tree structured objects. The usefulness of this method is illustrated in the context of detecting unexpected features in a data set of very large trees. The major contribution is a novel two-dimensional graphical representation of each tree, with a covariate coded by color.

The motivating data set contains three dimensional representations of brain artery systems of 105 subjects. Due to inaccuracies inherent in the medical imaging techniques, issues with the reconstruction algorithms and inconsistencies introduced by manual adjustment, various discrepancies are present in the data. The proposed representation enables quick visual detection of the most common discrepancies. For our driving example, this tool led to the modification of 10% of the artery trees and deletion of 6.7%.

The benefits of our cleaning method are demonstrated through a statistical hypothesis test on the effects of aging on vessel structure. The data cleaning resulted in improved significance levels. Our second example analyses brain artery images of healthy patients and patients with brain tumor. Our visualization can identify tumor patients.

This paper introduces a version of the argmax continuous mapping theorem that applies to M-estimation problems in which the objective functions converge to a limiting process with multiple maximizers. The concept of the smallest maximizer of a function in the d-dimensional Skorohod space is introduced and its main properties are studied. The resulting continuous mapping theorem is applied to three problems arising in change-point regression analysis. Some of the results proved in connection to the d-dimensional Skorohod space are also of independent interest.

We establish uniform-in-bandwidth consistency for kernel-type estimators of the differential entropy. We consider two kernel-type estimators of Shannon’s entropy. As a consequence, an asymptotic 100% confidence interval of entropy is provided.

Logistic regression is widely used in medical studies to investigate the relationship between a binary response variable Y and a set of potential predictors X. The binary response may represent, for example, the occurrence of some outcome of interest (Y=1 if the outcome occurred and Y=0 otherwise). In this paper, we consider the problem of estimating the logistic regression model with a cure fraction. A sample of observations is said to contain a cure fraction when a proportion of the study subjects (the so-called cured individuals, as opposed to the susceptibles) cannot experience the outcome of interest. One problem arising then is that it is usually unknown who are the cured and the susceptible subjects, unless the outcome of interest has been observed. In this setting, a logistic regression analysis of the relationship between X and Y among the susceptibles is no more straightforward. We develop a maximum likelihood estimation procedure for this problem, based on the joint modeling of the binary response of interest and the cure status. We investigate the identifiability of the resulting model. Then, we establish the consistency and asymptotic normality of the proposed estimator, and we conduct a simulation study to investigate its finite-sample behavior.

We consider the problems of estimating and detecting an unknown function f depending on a multidimensional variable (for instance, an image) observed in the Gaussian white noise. It is assumed that f belongs to anisotropic Sobolev class. The case of a function of infinitely many variables is also considered. An asymptotic study (as the noise level tends to zero) of the estimation and detection problems is done. In connection with the estimation problem, we construct asymptotically minimax estimators and establish sharp asymptotics for the minimax integrated squared risk. In the detection problem, we construct asymptotically minimax tests and provide conditions for distinguishability in the problem.

This paper investigates the problem of testing for linear Granger causality in mean when the number of parameters is high with the possible presence of nonlinear dynamics. Dependent innovations are taken into account by considering tests which asymptotic distributions is a weighted sum of chi-squares and tests with modified weight matrices. Wald, Lagrange Multiplier (LM) and Likelihood Ratio (LR) tests for linear causality in mean are studied. It is found that the LM tests based on restricted estimators significantly improve the analysis of linear Granger causality in mean relations when the dimension is high or when the autoregressive order is large. We also see that the tests based on a modified asymptotic distribution have a better control of the error of first kind when compared to the tests with modified statistic in finite samples. An application to international finance data is proposed to illustrate the robustness to the presence of nonlinearities of the studied tests.

We consider the problem of interaction neighborhood estimation from the partial observation of a finite number of realizations of a random field. We introduce a model selection rule to choose estimators of conditional probabilities among natural candidates. Our main result is an oracle inequality satisfied by the resulting estimator. We use then this selection rule in a two-step procedure to evaluate the interacting neighborhoods. The selection rule selects a small prior set of possible interacting points and a cutting step remove from this prior set the irrelevant points.

We also prove that the Ising models satisfy the assumptions of the main theorems, without restrictions on the temperature, on the structure of the interacting graph or on the range of the interactions. It provides therefore a large class of applications for our results. We give a computationally efficient procedure in these models. We finally show the practical efficiency of our approach in a simulation study.

We introduce variational Bayes methods for fast approximate inference in functional regression analysis. Both the standard cross-sectional and the increasingly common longitudinal settings are treated. The methodology allows Bayesian functional regression analyses to be conducted without the computational overhead of Monte Carlo methods. Confidence intervals of the model parameters are obtained both using the approximate variational approach and nonparametric resampling of clusters. The latter approach is possible because our variational Bayes functional regression approach is computationally efficient. A simulation study indicates that variational Bayes is highly accurate in estimating the parameters of interest and in approximating the Markov chain Monte Carlo-sampled joint posterior distribution of the model parameters. The methods apply generally, but are motivated by a longitudinal neuroimaging study of multiple sclerosis patients. Code used in simulations is made available as a web-supplement.

We present a simple way for calculating confidence intervals for a class of scalar functions of the parameters in least squares estimation when there are linear together with a small number of non-linear terms. We do not assume normality.

A commonly used semiparametric partial linear model is considered. We propose analyzing this model using a difference based approach. The procedure estimates the linear component based on the differences of the observations and then estimates the nonparametric component by either a kernel or a wavelet thresholding method using the residuals of the linear fit. It is shown that both the estimator of the linear component and the estimator of the nonparametric component asymptotically perform as well as if the other component were known. The estimator of the linear component is asymptotically efficient and the estimator of the nonparametric component is asymptotically rate optimal. A test for linear combinations of the regression coefficients of the linear component is also developed. Both the estimation and the testing procedures are easily implementable. Numerical performance of the procedure is studied using both simulated and real data. In particular, we demonstrate our method in an analysis of an attitude data set.

Variables in large data sets in biology or e-commerce often have a head, made up of very frequent values and a long tail of ever rarer values. Models such as the Zipf or Zipf–Mandelbrot provide a good description. The problem we address here is the visualization of two such long-tailed variables, as one might see in a bivariate Zipf context. We introduce a copula plot to display the joint behavior of such variables. The plot uses an empirical ordering of the data; we prove that this ordering is asymptotically accurate in a Zipf–Mandelbrot–Poisson model. We often see an association between entities at the head of one variable with those from the tail of the other. We present two generative models (saturation and bipartite preferential attachment) that show such qualitative behavior and we characterize the power law behavior of the marginal distributions in these models.

While many efforts have been made to prove that the Lasso behaves like a variable selection procedure at the price of strong (though unavoidable) assumptions on the geometric structure of these variables, much less attention has been paid to the oracle inequalities for the Lasso involving the ℓ₁-norm of the target vector. Such inequalities proved in the literature show that, provided that the regularization parameter is properly chosen, the Lasso approximately mimics the deterministic Lasso. Some of them do not require any assumption at all, neither on the structure of the variables nor on the regression function. Our first purpose here is to provide a conceptually very simple result in this direction in the framework of Gaussian models with non-random regressors.

Our second purpose is to propose a new estimator particularly adapted to deal with infinite countable dictionaries. This estimator is constructed as an ℓ₀-penalized estimator among a sequence of Lasso estimators associated to a dyadic sequence of growing truncated dictionaries. The selection procedure is choosing automatically the best level of truncation of the dictionary so as to make the best tradeoff between approximation, ℓ₁-regularization and sparsity. From a theoretical point of view, we shall provide an oracle inequality satisfied by this selected Lasso estimator.

The oracle inequalities presented in this paper are obtained via the application of a general theorem of model selection among a collection of nonlinear models which is a direct consequence of the Gaussian concentration inequality. The key idea that enables us to apply this general theorem is to see ℓ₁-regularization as a model selection procedure among ℓ₁-balls.

We revisit the adaptive Lasso as well as the thresholded Lasso with refitting, in a high-dimensional linear model, and study prediction error, ℓ_q-error (q∈{1,2}), and number of false positive selections. Our theoretical results for the two methods are, at a rather fine scale, comparable. The differences only show up in terms of the (minimal) restricted and sparse eigenvalues, favoring thresholding over the adaptive Lasso. As regards prediction and estimation, the difference is virtually negligible, but our bound for the number of false positives is larger for the adaptive Lasso than for thresholding. We also study the adaptive Lasso under beta-min conditions, which are conditions on the size of the coefficients. We show that for exact variable selection, the adaptive Lasso generally needs more severe beta-min conditions than thresholding. Both the two-stage methods add value to the one-stage Lasso in the sense that, under appropriate restricted and sparse eigenvalue conditions, they have similar prediction and estimation error as the one-stage Lasso but substantially less false positives. Regarding the latter, we provide a lower bound for the Lasso with respect to false positive selections.

The aim of this paper is to provide a comprehensive introduction for the study of ℓ₁-penalized estimators in the context of dependent observations. We define a general ℓ₁-penalized estimator for solving problems of stochastic optimization. This estimator turns out to be the LASSO [Tib96] in the regression estimation setting. Powerful theoretical guarantees on the statistical performances of the LASSO were provided in recent papers, however, they usually only deal with the iid case. Here, we study this estimator under various dependence assumptions.

We consider in this paper the multivariate regression problem, when the target regression matrix A is close to a low rank matrix. Our primary interest is in on the practical case where the variance of the noise is unknown. Our main contribution is to propose in this setting a criterion to select among a family of low rank estimators and prove a non-asymptotic oracle inequality for the resulting estimator. We also investigate the easier case where the variance of the noise is known and outline that the penalties appearing in our criterions are minimal (in some sense). These penalties involve the expected value of Ky-Fan norms of some random matrices. These quantities can be evaluated easily in practice and upper-bounds can be derived from recent results in random matrix theory.

Time series models are often constructed by combining nonstationary effects such as trends with stochastic processes that are believed to be stationary. Although stationarity of the underlying process is typically crucial to ensure desirable properties or even validity of statistical estimators, there are numerous time series models for which this stationarity is not yet proven. A major barrier is that the most commonly-used methods assume φ-irreducibility, a condition that can be violated for the important class of discrete-valued observation-driven models.

We show (strict) stationarity for the class of Generalized Autoregressive Moving Average (GARMA) models, which provides a flexible analogue of ARMA models for count, binary, or other discrete-valued data. We do this from two perspectives. First, we show conditions under which GARMA models have a unique stationary distribution (so are strictly stationary when initialized in that distribution). This result potentially forms the foundation for broadly showing consistency and asymptotic normality of maximum likelihood estimators for GARMA models. Since these conclusions are not immediate, however, we also take a second approach. We show stationarity and ergodicity of a perturbed version of the GARMA model, which utilizes the fact that the perturbed model is φ-irreducible and immediately implies consistent estimation of the mean, lagged covariances, and other functionals of the perturbed process. We relate the perturbed and original processes by showing that the perturbed model yields parameter estimates that are arbitrarily close to those of the original model.

We focus on the performances of tree-structured wavelet estimators belonging to a large family of keep-or-kill rules, namely the Vertical Block Thresholding family. For each estimator, we provide the maximal functional space (maxiset) for which the quadratic risk reaches a given rate of convergence. Following a discussion on the maxiset embeddings, we identify the ideal estimator of this family, that is the one associated with the largest maxiset. We emphasize the importance of such a result since the ideal estimator is different from the usual (plug-in) estimator used to mimic the performances of the Oracle. Finally, we confirm the good performances of the ideal estimator compared to the other elements of that family through extensive numerical experiments.

This paper considers nonparametric regression models with long memory errors and predictors. Unlike in weak dependence situations, we show that the estimation of the conditional mean has influence on the estimation of both, the conditional variance and the error density. In particular, the estimation of the conditional mean has a negative effect on the asymptotic behaviour of the conditional variance estimator. On the other hand, surprisingly, estimation of the conditional mean may reduce convergence rates of the residual-based Parzen-Rosenblatt density estimator, as compared to the errors-based one. Our asymptotic results reveal small/large bandwidth dichotomous behaviour. In particular, we present a method which guarantees that a chosen bandwidth implies standard weakly dependent-type asymptotics. Our results are confirmed by an extensive simulation study. Furthermore, our theoretical lemmas may be used in different problems related to nonparametric regression with long memory, like cross-validation properties, bootstrap, goodness-of-fit or quadratic forms.

Respondent-driven sampling is a widely-used network sampling technique, designed to sample from hard-to-reach populations. Estimation from the resulting samples is an area of active research, with software available to compute at least four estimators of a population proportion. Each estimator is claimed to address deficiencies in previous estimators, however those claims are often unsubstantiated. In this study we provide a simulation-based comparison of five existing estimators, focusing on sampling conditions which a recent estimator is designed to address. We find no estimator consistently out-performs all others, and highlight sampling conditions in which each is to be preferred.

Given i.i.d. observations of a random vector X∈ℝ^p, we study the problem of estimating both its covariance matrix Σ^*, and its inverse covariance or concentration matrix Θ^*=(Σ^*)⁻¹. When X is multivariate Gaussian, the non-zero structure of Θ^* is specified by the graph of an associated Gaussian Markov random field; and a popular estimator for such sparse Θ^* is the ℓ₁-regularized Gaussian MLE. This estimator is sensible even for for non-Gaussian X, since it corresponds to minimizing an ℓ₁-penalized log-determinant Bregman divergence. We analyze its performance under high-dimensional scaling, in which the number of nodes in the graph p, the number of edges s, and the maximum node degree d, are allowed to grow as a function of the sample size n. In addition to the parameters (p,s,d), our analysis identifies other key quantities that control rates: (a) the ℓ_∞-operator norm of the true covariance matrix Σ^*; and (b) the ℓ_∞ operator norm of the sub-matrix Γ^*_SS, where S indexes the graph edges, and Γ^*=(Θ^*)⁻¹⊗(Θ^*)⁻¹; and (c) a mutual incoherence or irrepresentability measure on the matrix Γ^* and (d) the rate of decay 1/f(n,δ) on the probabilities {|Σ̂ⁿ_ij−Σ^*_ij|>δ}, where Σ̂ⁿ is the sample covariance based on n samples. Our first result establishes consistency of our estimate Θ̂ in the elementwise maximum-norm. This in turn allows us to derive convergence rates in Frobenius and spectral norms, with improvements upon existing results for graphs with maximum node degrees $d=o(\sqrt{s})$. In our second result, we show that with probability converging to one, the estimate Θ̂ correctly specifies the zero pattern of the concentration matrix Θ^*. We illustrate our theoretical results via simulations for various graphs and problem parameters, showing good correspondences between the theoretical predictions and behavior in simulations.

Standard Gaussian graphical models implicitly assume that the conditional independence among variables is common to all observations in the sample. However, in practice, observations are usually collected from heterogeneous populations where such an assumption is not satisfied, leading in turn to nonlinear relationships among variables. To address such situations we explore mixtures of Gaussian graphical models; in particular, we consider both infinite mixtures and infinite hidden Markov models where the emission distributions correspond to Gaussian graphical models. Such models allow us to divide a heterogeneous population into homogenous groups, with each cluster having its own conditional independence structure. As an illustration, we study the trends in foreign exchange rate fluctuations in the pre-Euro era.

The Huber’s Criterion is a useful method for robust regression. The adaptive least absolute shrinkage and selection operator (lasso) is a popular technique for simultaneous estimation and variable selection. The adaptive weights in the adaptive lasso allow to have the oracle properties. In this paper we propose to combine the Huber’s criterion and adaptive penalty as lasso. This regression technique is resistant to heavy-tailed errors or outliers in the response. Furthermore, we show that the estimator associated with this procedure enjoys the oracle properties. This approach is compared with LAD-lasso based on least absolute deviation with adaptive lasso. Extensive simulation studies demonstrate satisfactory finite-sample performance of such procedure. A real example is analyzed for illustration purposes.

A new class of statistical deformable models is introduced to study high-dimensional curves or images. In addition to the standard measurement error term, these deformable models include an extra error term modeling the individual variations in intensity around a mean pattern. It is shown that an appropriate tool for statistical inference in such models is the notion of sample Fréchet means, which leads to estimators of the deformation parameters and the mean pattern. The main contribution of this paper is to study how the behavior of these estimators depends on the number n of design points and the number J of observed curves (or images). Numerical experiments are given to illustrate the finite sample performances of the procedure.

This paper concerns the use of the expectation-maximisation (EM) algorithm for inference in partially observed diffusion processes. In this context, a well known problem is that all except a few diffusion processes lack closed-form expressions of the transition densities. Thus, in order to estimate efficiently the EM intermediate quantity we construct, using novel techniques for unbiased estimation of diffusion transition densities, a random weight fixed-lag auxiliary particle smoother, which avoids the well known problem of particle trajectory degeneracy in the smoothing mode. The estimator is justified theoretically and demonstrated on a simulated example.

The performance of principal component analysis suffers badly in the presence of outliers. This paper proposes two novel approaches for robust principal component analysis based on semidefinite programming. The first method, maximum mean absolute deviation rounding, seeks directions of large spread in the data while damping the effect of outliers. The second method produces a low-leverage decomposition of the data that attempts to form a low-rank model for the data by separating out corrupted observations. This paper also presents efficient computational methods for solving these semidefinite programs. Numerical experiments confirm the value of these new techniques.

In this paper we consider the trace regression model. Assume that we observe a small set of entries or linear combinations of entries of an unknown matrix A₀ corrupted by noise. We propose a new rank penalized estimator of A₀. For this estimator we establish general oracle inequality for the prediction error both in probability and in expectation. We also prove upper bounds for the rank of our estimator. Then, we apply our general results to the problems of matrix completion and matrix regression. In these cases our estimator has a particularly simple form: it is obtained by hard thresholding of the singular values of a matrix constructed from the observations.

We consider a linear regression problem in a high dimensional setting where the number of covariates p can be much larger than the sample size n. In such a situation, one often assumes sparsity of the regression vector, i.e., the regression vector contains many zero components. We propose a Lasso-type estimator β̂^Quad (where ‘Quad’ stands for quadratic) which is based on two penalty terms. The first one is the ℓ₁ norm of the regression coefficients used to exploit the sparsity of the regression as done by the Lasso estimator, whereas the second is a quadratic penalty term introduced to capture some additional information on the setting of the problem. We detail two special cases: the Elastic-Net β̂^EN introduced in [42], which deals with sparse problems where correlations between variables may exist; and the Smooth-Lasso β̂^SL, which responds to sparse problems where successive regression coefficients are known to vary slowly (in some situations, this can also be interpreted in terms of correlations between successive variables). From a theoretical point of view, we establish variable selection consistency results and show that β̂^Quad achieves a Sparsity Inequality, i.e., a bound in terms of the number of non-zero components of the ‘true’ regression vector. These results are provided under a weaker assumption on the Gram matrix than the one used by the Lasso. In some situations this guarantees a significant improvement over the Lasso. Furthermore, a simulation study is conducted and shows that the S-Lasso β̂^SL performs better than known methods as the Lasso, the Elastic-Net β̂^EN, and the Fused-Lasso (introduced in [30]) with respect to the estimation accuracy. This is especially the case when the regression vector is ‘smooth’, i.e., when the variations between successive coefficients of the unknown parameter of the regression are small. The study also reveals that the theoretical calibration of the tuning parameters and the one based on 10 fold cross validation imply two S-Lasso solutions with close performance.

Simultaneous tracking of multiple moving objects extracted from an image sequence is an important problem which finds numerous applications in science and engineering. In this article we conduct an investigation of the theoretical properties a statistical model for tracking such moving objects, or targets. This tracking model allows for birth, death, splitting and merging of targets, and uses a Markov model to decide the times at which such events occur. This model also assumes that the track traveled by each target behaves like a Gaussian process. The estimated tracking solution is obtained via maximum likelihood. One of the contributions of this article is to establish the almost sure consistency to the data association problem by using these maximum likelihood tracking estimates. A major technical challenge for proving this consistency result is to identify the correct track (data association) amongst a group of similar (but incorrect) track proposals that are results of various combinations of target birth, death, splitting and/or merging. This consistency property of the tracking estimates is empirically verified by numerical experiments. To the best of our knowledge, this is the first time that a comprehensive study is performed for the large sample properties of a multiple target tracking method. In addition, the issue of how to quantify the confidence of a tracking estimate is also addressed.

In this paper we investigate the geometry of a discrete Bayesian network whose graph is a tree all of whose variables are binary and the only observed variables are those labeling its leaves. We provide the full geometric description of these models which is given by a set of polynomial equations together with a set of complementary implied inequalities induced by the positivity of probabilities on hidden variables. The phylogenetic invariants given by the equations can be useful in the construction of simple diagnostic tests. However, in this paper we point out the importance of also incorporating the associated inequalities into any statistical analysis. The full characterization of these inequality constraints derived in this paper helps us determine how and why routine statistical methods can break down for this model class.

We estimate the unknown parameters of an asymmetric bifurcating autoregressive process (BAR) when some of the data are missing. In this aim, we model the observed data by a two-type Galton-Watson process consistent with the binary tree structure of the data. Under independence between the process leading to the missing data and the BAR process and suitable assumptions on the driven noise, we establish the strong consistency of our estimators on the set of non-extinction of the Galton-Watson process, via a martingale approach. We also prove a quadratic strong law and the asymptotic normality.

A pivotal problem in Bayesian nonparametrics is the construction of prior distributions on the space M(V) of probability measures on a given domain V. In principle, such distributions on the infinite-dimensional space M(V) can be constructed from their finite-dimensional marginals—the most prominent example being the construction of the Dirichlet process from finite-dimensional Dirichlet distributions. This approach is both intuitive and applicable to the construction of arbitrary distributions on M(V), but also hamstrung by a number of technical difficulties. We show how these difficulties can be resolved if the domain V is a Polish topological space, and give a representation theorem directly applicable to the construction of any probability distribution on M(V) whose first moment measure is well-defined. The proof draws on a projective limit theorem of Bochner, and on properties of set functions on Polish spaces to establish countable additivity of the resulting random probabilities.

Let X be a 1-dimensional diffusion process. We study a simple class of estimators, which rely only on one sample data $\{X_{\frac{i}{n}},0\leq i\leq nt\}$, for the occupation time ∫₀^tI_A(X_s)ds of process X in some set A. The main concern of this paper is the rates of convergence of the estimators. First, we consider the case that A is a finite union of some intervals in ℝ, then we show that the estimator converges at rate n^−3/4. Second, we consider the so-called stochastic corridor in mathematical finance. More precisely, we let A be a stochastic interval, say [X_t0,∞) for some t₀∈(0,t), then we show that the estimator converges at rate n^−1/2. Some discussions about the exactness of the rates are also presented.

Recently, Chazal, Cohen-Steiner and Mérigot have defined a distance function to measures to answer geometric inference problems in a probabilistic setting. According to their result, the topological properties of a shape can be recovered by using the distance to a known measure ν, if ν is close enough to a measure μ concentrated on this shape. Here, close enough means that the Wasserstein distance W₂ between μ and ν is sufficiently small. Given a point cloud, a natural candidate for ν is the empirical measure μ_n. Nevertheless, in many situations the data points are not located on the geometric shape but in the neighborhood of it, and μ_n can be too far from μ. In a deconvolution framework, we consider a slight modification of the classical kernel deconvolution estimator, and we give a consistency result and rates of convergence for this estimator. Some simulated experiments illustrate the deconvolution method and its application to geometric inference on various shapes and with various noise distributions.

This paper considers the problem of estimation in a general semiparametric regression model when error-prone covariates are modeled parametrically while covariates measured without error are modeled nonparametrically. To account for the effects of measurement error, we apply a correction to a criterion function. The specific form of the correction proposed allows Monte Carlo simulations in problems for which the direct calculation of a corrected criterion is difficult. Therefore, in contrast to methods that require solving integral equations of possibly multiple dimensions, as in the case of multiple error-prone covariates, we propose methodology which offers a simple implementation. The resulting methods are functional, they make no assumptions about the distribution of the mismeasured covariates. We utilize profile kernel and backfitting estimation methods and derive the asymptotic distribution of the resulting estimators. Through numerical studies we demonstrate the applicability of proposed methods to Poisson, logistic and multivariate Gaussian partially linear models. We show that the performance of our methods is similar to a computationally demanding alternative. Finally, we demonstrate the practical value of our methods when applied to Nevada Test Site (NTS) Thyroid Disease Study data.

The conjugate binomial and Poisson models are commonly used for estimating proportions or rates. However, it is not well known that the conventional noninformative conjugate priors tend to shrink the posterior quantiles toward the boundary or toward the middle of the parameter space, making them thus appear excessively informative. The shrinkage is always largest when the number of observed events is small. This behavior persists for all sample sizes and exposures. The effect of the prior is therefore most conspicuous and potentially controversial when analyzing rare events. As alternative default conjugate priors, I introduce Beta(1/3, 1/3) and Gamma(1/3, 0), which I call ‘neutral’ priors because they lead to posterior distributions with approximately 50 per cent probability that the true value is either smaller or larger than the maximum likelihood estimate. This holds for all sample sizes and exposures as long as the point estimate is not at the boundary of the parameter space. I also discuss the construction of informative prior distributions. Under the suggested formulation, the posterior median coincides approximately with the weighted average of the prior median and the sample mean, yielding priors that perform more intuitively than those obtained by matching moments and quantiles.

The goal of this article is to introduce an iterative application of dimension reduction methods. It is known that in some situations, methods such as Sliced Inverse Regression (SIR), Ordinary Least Squares (OLS) and Cumulative Mean Estimation (CUME) are able to find only a partial basis for the dimension reduction subspace. However, for many models these methods are very good estimators of this partial basis. In this paper we propose a simple iterative procedure which differs from existing combined approaches in the sense that the initial partial basis is estimated first and the second dimension reduction approach seeks only the remainder of the dimension reduction subspace. Our approach is compared against that of existing combined dimension reduction approaches via simulated data as well as two example data sets.

We study the frequentist risk performance of Bayesian estimators of a bounded location parameter, and focus on conditions placed on the shape of the prior density guaranteeing dominance over the minimum risk equivariant (MRE) estimator. For a large class of even and logconcave densities, even convex loss functions, we demonstrate in a unified manner that symmetric priors which are bowled shaped and logconcave lead to Bayesian dominating estimators. The results generalize similar results obtained by Marchand and Strawderman for the fully uniform prior, as well as those obtained by Kubokawa for squared error loss. Finally, we present a detailed example and several remarks.

We study an extension to general Markov random fields of the resampling scheme given in Bickel and Levina (2006) [4] for texture synthesis with stationary Markov mesh models. The procedure generates bootstrap replicates of a sample using kernel regression and the principle of Gibbs sampling. Consistency of the bootstrap distribution is investigated under the Dobrushin contraction condition. Some simulation examples are given, in particular for the texture synthesis, for which the multiscale algorithm of Paget and Longstaff (1998) [27] is revisited.

In the context of clustering, we assume a generative model where each cluster is the result of sampling points in the neighborhood of an embedded smooth surface; the sample may be contaminated with outliers, which are modeled as points sampled in space away from the clusters. We consider a prototype for a higher-order spectral clustering method based on the residual from a local linear approximation. We obtain theoretical guarantees for this algorithm and show that, in terms of both separation and robustness to outliers, it outperforms the standard spectral clustering algorithm (based on pairwise distances) of Ng, Jordan and Weiss (NIPS ’01). The optimal choice for some of the tuning parameters depends on the dimension and thickness of the clusters. We provide estimators that come close enough for our theoretical purposes. We also discuss the cases of clusters of mixed dimensions and of clusters that are generated from smoother surfaces. In our experiments, this algorithm is shown to outperform pairwise spectral clustering on both simulated and real data.

This paper focuses on recovering an unknown vector β from the noisy data Y=Xβ+σξ, where X is a known n×p-matrix, ξ is a standard white Gaussian noise, and σ is an unknown noise level. In order to estimate β, a spectral regularization method is used, and our goal is to choose its regularization parameter with the help of the data Y. In this paper, we deal solely with regularization methods based on the so-called ordered smoothers (see [13]) and extend the oracle inequality from [11] to the case, where the noise level is unknown.

The problem of estimating a conditional density is considered. Given a collection of partitions, we propose a procedure that selects from the data the best partition among that collection and then provides the best piecewise polynomial estimator built on that partition. The observations are not supposed to be independent but only β-mixing; in particular, our study includes the estimation of the transition density of a Markov chain. For a well-chosen collection of possibly irregular partitions, we obtain oracle-type inequalities and adaptivity results in the minimax sense over a wide range of possibly anisotropic and inhomogeneous Besov classes. We end with a short simulation study.

We introduce the concept of penalized wavelets to facilitate seamless embedding of wavelets into semiparametric regression models. In particular, we show that penalized wavelets are analogous to penalized splines; the latter being the established approach to function estimation in semiparametric regression. They differ only in the type of penalization that is appropriate. This fact is not borne out by the existing wavelet literature, where the regression modelling and fitting issues are overshadowed by computational issues such as efficiency gains afforded by the Discrete Wavelet Transform and partially obscured by a tendency to work in the wavelet coefficient space. With penalized wavelet structure in place, we then show that fitting and inference can be achieved via the same general approaches used for penalized splines: penalized least squares, maximum likelihood and best prediction within a frequentist mixed model framework, and Markov chain Monte Carlo and mean field variational Bayes within a Bayesian framework. Penalized wavelets are also shown have a close relationship with wide data (“p≫n”) regression and benefit from ongoing research on that topic.

We consider the problem of testing for zero variance components in linear mixed models with correlated or heteroscedastic errors. In the case of independent and identically distributed errors, a valid test exists, which is based on the exact finite sample distribution of the restricted likelihood ratio test statistic under the null hypothesis. We propose to make use of a transformation to derive the (approximate) null distribution for the restricted likelihood ratio test statistic in the case of a general error covariance structure. The method can also be applied in the case of testing for a random effect in linear mixed models with several random effects by writing the model as one with a single random effect and a more complex covariance structure. The proposed test proves its value in simulations and is finally applied to an interesting question in the field of well-being economics.

We consider the efficient estimation of the semiparametric additive transformation model with current status data. A wide range of survival models and econometric models can be incorporated into this general transformation framework. We apply the B-spline approach to simultaneously estimate the linear regression vector, the nondecreasing transformation function, and a set of nonparametric regression functions. We show that the parametric estimate is semiparametric efficient in the presence of multiple nonparametric nuisance functions. An explicit consistent B-spline estimate of the asymptotic variance is also provided. All nonparametric estimates are smooth, and shown to be uniformly consistent and have faster than cubic rate of convergence. Interestingly, we observe the convergence rate interfere phenomenon, i.e., the convergence rates of B-spline estimators are all slowed down to equal the slowest one. The constrained optimization is not required in our implementation. Numerical results are used to illustrate the finite sample performance of the proposed estimators.

We develop a self-normalization (SN) based test to test the structural stability of temporally dependent functional observations. Testing for a change point in the mean of functional data has been studied in Berkes, Gabrys, Horváth and Kokoszka [4], but their test was developed under the independence assumption. In many applications, functional observations are expected to be dependent, especially when the data is collected over time. Building on the SN-based change point test proposed in Shao and Zhang [23] for a univariate time series, we extend the SN-based test to the functional setup by testing the constant mean of the finite dimensional eigenvectors after performing functional principal component analysis. Asymptotic theories are derived under both the null and local alternatives. Through theory and extensive simulations, our SN-based test statistic proposed in the functional setting is shown to inherit some useful properties in the univariate setup: the test is asymptotically distribution free and its power is monotonic. Furthermore, we extend the SN-based test to identify potential change points in the dependence structure of functional observations. The method is then applied to central England temperature series to detect the warming trend and to gridded temperature fields generated by global climate models to test for changes in spatial bias structure over time.

We study three estimators for the interval censoring case 2 problem, a histogram-type estimator, proposed in [1], the maximum likelihood estimator (MLE) and the smoothed MLE, using a smoothing kernel. Our focus is on the asymptotic distribution of the estimators at a fixed point. The estimators are compared in a simulation study.

The Pearson’s chi-square and the log-likelihood ratio chi-square statistics are fundamental tools in multinomial goodness-of-fit testing. Cressie and Read (1984) constructed a general family of divergences which includes both statistics as special cases. This family is indexed by a single real parameter. Divergences at one end of the scale are powerful against deviations of one type while being poor against deviations of the other type. The reverse property holds for divergences at the other end of the scale. Several other families of divergences available in the literature also show similar behavior. We present several inlier control techniques in the context of multinomial goodness-of-fit testing which generate procedures having reasonably high powers for both kinds of alternatives. We explain the motivation behind the construction of the inlier modified test statistics, establish the asymptotic null distribution of the inlier modified statistics and explore their performance through simulation and real data examples to substantiate the theory developed.

We study the distribution of hard-, soft-, and adaptive soft-thresholding estimators within a linear regression model where the number of parameters k can depend on sample size n and may diverge with n. In addition to the case of known error-variance, we define and study versions of the estimators when the error-variance is unknown. We derive the finite-sample distribution of each estimator and study its behavior in the large-sample limit, also investigating the effects of having to estimate the variance when the degrees of freedom n−k does not tend to infinity or tends to infinity very slowly. Our analysis encompasses both the case where the estimators are tuned to perform consistent variable selection and the case where the estimators are tuned to perform conservative variable selection. Furthermore, we discuss consistency, uniform consistency and derive the uniform convergence rate under either type of tuning.

In a linear model relevance of a categorical predictor with ordered levels is typically tested by use of the standard F-test (known from statistical textbooks). Such a test can also be applied for testing whether the regression function is linear in the ordinal predictor’s class labels. In this paper we propose an alternative (restricted) likelihood ratio test for these hypotheses which is especially suited for ordinal predictors and is based on the mixed model formulation of penalized dummy coefficients. We show in simulation studies that the new test is more powerful than the standard F-test in many situations. The advantage of the new test is especially striking when the number of ordered levels is moderate or large. Using the relationship to mixed effect models and robust existent fitting software obtaining the test and its null distribution is very fast; a fast R implementation is provided.

We consider the problem of functional principal component analysis for correlated functional data. In particular, we focus on a separable covariance structure and consider irregularly and possibly sparsely observed sample trajectories. By observing that under the sparse measurements setting, the empirical covariance of pre-smoothed sample trajectories is a highly biased estimator along the diagonal, we propose to modify the empirical covariance by estimating the diagonal and off-diagonal parts of the covariance kernel separately. We prove that under a separable covariance structure, this method can consistently estimate the eigenfunctions of the covariance kernel. We also quantify the role of the correlation in the L² risk of the estimator, and show that under a weak correlation regime, the risk achieves the optimal nonparametric rate when the number of measurements per curve is bounded.

We consider a compound Poisson process with symmetric Bernoulli jumps, observed at times iΔ for i=0,1,… over [0,T], for different sizes of Δ=Δ_T relative to T in the limit T→∞. We quantify the smooth statistical transition from a microscopic Poissonian regime (when Δ_T→0) to a macroscopic Gaussian regime (when Δ_T→∞). The classical quadratic variation estimator is efficient for estimating the intensity of the Poisson process in both microscopic and macroscopic scales but surprisingly, it shows a substantial loss of information in the intermediate scale Δ_T→Δ_∞∈(0,∞). This loss can be explicitly related to Δ_∞. We provide an estimator that is efficient simultaneously in microscopic, intermediate and macroscopic regimes. We discuss the implications of these findings beyond this idealised framework.