We consider projection methods for the estimation of the cumulative distribution function under interval censoring, case 1. Such censored data also known as current status data, arise when the only information available on the variable of interest is whether it is greater or less than an observed random time. Two types of adaptive estimators are investigated. The first one is a two-step estimator built as a quotient estimator. The second estimator results from a mean square regression contrast. Both estimators are proved to achieve automatically the standard optimal rate associated with the unknown regularity of the function, but with some restriction for the quotient estimator. Simulation experiments are presented to illustrate and compare the methods.

We consider independent random variables (r.v.’s) with a common mean μ that either satisfy Lindeberg’s condition, or are symmetric around μ. Present forms of existing functional central limit theorems (FCLT’s) for Studentized partial sums of such r.v.’s on D[0,1] are seen to be of some use for constructing asymptotic confidence intervals, or what we call functional asymptotic confidence intervals (FACI’s), for μ. In this paper we establish completely data-based versions of these FCLT’s and thus extend their applicability in this regard. Two special examples of new FACI’s for μ are presented.

Suppose that we observe independent random pairs (X₁,Y₁), (X₂,Y₂), …, (X_n,Y_n). Our goal is to estimate regression functions such as the conditional mean or β–quantile of Y given X, where 0<β<1. In order to achieve this we minimize criteria such as, for instance, $$\sum_{i=1}^n ρ(f(X_i) - Y_i) + λ⋅\mathrm{TV}(f)$$ among all candidate functions f. Here ρ is some convex function depending on the particular regression function we have in mind, TV(f) stands for the total variation of f, and λ>0 is some tuning parameter. This framework is extended further to include binary or Poisson regression, and to include localized total variation penalties. The latter are needed to construct estimators adapting to inhomogeneous smoothness of f. For the general framework we develop noniterative algorithms for the solution of the minimization problems which are closely related to the taut string algorithm (cf. Davies and Kovac, 2001). Further we establish a connection between the present setting and monotone regression, extending previous work by Mammen and van de Geer (1997). The algorithmic considerations and numerical examples are complemented by two consistency results.

In this paper, we consider the problem of partitioning a small data sample drawn from a mixture of k product distributions. We are interested in the case that individual features are of low average quality γ, and we want to use as few of them as possible to correctly partition the sample. We analyze a spectral technique that is able to approximately optimize the total data size—the product of number of data points n and the number of features K—needed to correctly perform this partitioning as a function of 1/γ for K>n. Our goal is motivated by an application in clustering individuals according to their population of origin using markers, when the divergence between any two of the populations is small.

We investigate the asymptotic normality of the posterior distribution in the discrete setting, when model dimension increases with sample size. We consider a probability mass function θ₀ on ℕ∖{0} and a sequence of truncation levels (k_n)_n satisfying k_n³≤ninf _i≤knθ₀(i). Let θ̂ denote the maximum likelihood estimate of (θ₀(i))_i≤kn and let Δ_n(θ₀) denote the k_n-dimensional vector which i-th coordinate is defined by $\sqrt{n}(\hat{\theta}_{n}(i)-\theta_{0}(i))$ for 1≤i≤k_n. We check that under mild conditions on θ₀ and on the sequence of prior probabilities on the k_n-dimensional simplices, after centering and rescaling, the variation distance between the posterior distribution recentered around θ̂_n and rescaled by $\sqrt{n}$ and the k_n-dimensional Gaussian distribution $\mathcal{N}(\Delta_{n}(\theta_{0}),I^{-1}(\theta_{0}))$ converges in probability to 0. This theorem can be used to prove the asymptotic normality of Bayesian estimators of Shannon and Rényi entropies.

The proofs are based on concentration inequalities for centered and non-centered Chi-square (Pearson) statistics. The latter allow to establish posterior concentration rates with respect to Fisher distance rather than with respect to the Hellinger distance as it is commonplace in non-parametric Bayesian statistics.

The Commission of the European Union, as well the United States Environmental Protection Agency, have set limit values for some pollutants in the ambient air that have been shown to have adverse effects on human and environmental health. It is therefore important to identify regions where the probability of exceeding those limits is high. We propose a two-step procedure for estimating the probability of exceeding the legal limits that combines smoothing in the time domain with spatial interpolation. For illustration, we show an application to particulate matter with diameter less than 10 microns (PM₁₀) in the North-Italian region Piemonte.

We prove an exponential inequality for negatively associated and strictly stationary random variables. A condition is given for almost sure convergence and the associated rate of convergence is specified in terms of the underlying covariance function.

Model selection is often performed by empirical risk minimization. The quality of selection in a given situation can be assessed by risk bounds, which require assumptions both on the margin and the tails of the losses used. Starting with examples from the 3 basic estimation problems, regression, classification and density estimation, we formulate risk bounds for empirical risk minimization and prove them at a very general level, for general margin and power tail behavior of the excess losses. These bounds we then apply to typical examples.

Our concern is selecting the concentration matrix’s nonzero coefficients for a sparse Gaussian graphical model in a high-dimensional setting. This corresponds to estimating the graph of conditional dependencies between the variables. We describe a novel framework taking into account a latent structure on the concentration matrix. This latent structure is used to drive a penalty matrix and thus to recover a graphical model with a constrained topology. Our method uses an ℓ₁ penalized likelihood criterion. Inference of the graph of conditional dependencies between the variates and of the hidden variables is performed simultaneously in an iterative EM-like algorithm named SIMoNe (Statistical Inference for Modular Networks). Performances are illustrated on synthetic as well as real data, the latter concerning breast cancer. For gene regulation networks, our method can provide a useful insight both on the mutual influence existing between genes, and on the modules existing in the network.

The aim of this paper is to compare three regularized particle filters in an online data processing context. We carry out the comparison in terms of hidden states filtering and parameter estimation, considering a Bayesian paradigm and a univariate Stochastic Volatility (SV) model. We discuss the use of an improper prior distribution in the initialization of the filtering procedure and show that the regularized Auxiliary Particle Filter (APF) outperforms the regularized Sequential Importance Sampling (SIS) and the regularized Sampling Importance Resampling (SIR).

When in a full exponential family the maximum likelihood estimate (MLE) does not exist, the MLE may exist in the Barndorff-Nielsen completion of the family. We propose a practical algorithm for finding the MLE in the completion based on repeated linear programming using the R contributed package rcdd and illustrate it with three generalized linear model examples. When the MLE for the null hypothesis lies in the completion, likelihood ratio tests of model comparison are almost unchanged from the usual case. Only the degrees of freedom need to be adjusted. When the MLE lies in the completion, confidence intervals are changed much more from the usual case. The MLE of the natural parameter can be thought of as having gone to infinity in a certain direction, which we call a generic direction of recession. We propose a new one-sided confidence interval which says how close to infinity the natural parameter may be. This maps to one-sided confidence intervals for mean values showing how close to the boundary of their support they may be.

Graphical models are used to describe the conditional independence relations in multivariate data. They have been used for a variety of problems, including log-linear models (Liu and Massam, 2006), network analysis (Holland and Leinhardt, 1981; Strauss and Ikeda, 1990; Wasserman and Pattison, 1996; Pattison and Wasserman, 1999; Robins et al., 1999), graphical Gaussian models (Roverato and Whittaker, 1998; Giudici and Green, 1999; Marrelec and Benali, 2006), and genetics (Dobra et al., 2004). A distribution that satisfies the conditional independence structure of a graph is Markov. A graphical model is a family of distributions that is restricted to be Markov with respect to a certain graph. In a Bayesian problem, one may specify a prior over the graphical model. Such a prior is called a hyper Markov law if the random marginals also satisfy the independence constraints. Previous work in this area includes (Dempster, 1972; Dawid and Lauritzen, 1993; Giudici and Green, 1999; Letac and Massam, 2007). We explore graphical models based on a non-parametric family of distributions, developed from Dirichlet processes.

SIMEX is a general-purpose technique for measurement error correction. There is a substantial literature on the application and theory of SIMEX for purely parametric problems, as well as for purely nonparametric regression problems, but there is neither application nor theory for semiparametric problems. Motivated by an example involving radiation dosimetry, we develop the basic theory for SIMEX in semiparametric problems using kernel-based estimation methods. This includes situations that the mismeasured variable is modeled purely parametrically, purely nonparametrically, or that the mismeasured variable has components that are modeled both parametrically and nonparametrically. Using our asymptotic expansions, easily computed standard error formulae are derived, as are the bias properties of the nonparametric estimator. The standard error method represents a new method for estimating variability of nonparametric estimators in semiparametric problems, and we show in both simulations and in our example that it improves dramatically on first order methods.

We find that for estimating the parametric part of the model, standard bandwidth choices of order O(n^−1/5) are sufficient to ensure asymptotic normality, and undersmoothing is not required. SIMEX has the property that it fits misspecified models, namely ones that ignore the measurement error. Our work thus also more generally describes the behavior of kernel-based methods in misspecified semiparametric problems.

This paper focuses on the problem of the estimation of the cumulative hazard function of a distribution on d-dimensional Euclidean space when the data points are subject to censoring by an arbitrary adapted random set. A problem involving observability of the estimator proposed in [8] and [9] is resolved and a functional central limit theorem is proven for the revised estimator. Several examples and applications are discussed, and the validity of bootstrap methods is established in each case.

Exact and explicit expressions for the variogram of first-order intrinsic autoregressions have not been known. Various asymptotic expansions and approximations have been used to compute the variogram. In this note, an exact and explicit expression applicable for all parameter values is derived. The expression involves Appell’s hypergeometric function of the fourth kind. Various particular cases of the expression are also derived.

This paper discusses a class of thresholding-based iterative selection procedures (TISP) for model selection and shrinkage. People have long before noticed the weakness of the convex l₁-constraint (or the soft-thresholding) in wavelets and have designed many different forms of nonconvex penalties to increase model sparsity and accuracy. But for a nonorthogonal regression matrix, there is great difficulty in both investigating the performance in theory and solving the problem in computation. TISP provides a simple and efficient way to tackle this so that we successfully borrow the rich results in the orthogonal design to solve the nonconvex penalized regression for a general design matrix. Our starting point is, however, thresholding rules rather than penalty functions. Indeed, there is a universal connection between them. But a drawback of the latter is its non-unique form, and our approach greatly facilitates the computation and the analysis. In fact, we are able to build the convergence theorem and explore theoretical properties of the selection and estimation via TISP nonasymptotically. More importantly, a novel Hybrid-TISP is proposed based on hard-thresholding and ridge-thresholding. It provides a fusion between the l₀-penalty and the l₂-penalty, and adaptively achieves the right balance between shrinkage and selection in statistical modeling. In practice, Hybrid-TISP shows superior performance in test-error and is parsimonious.

In this short note, we show how to use concentration inequalities in order to build exact confidence intervals for the Hurst parameter associated with a one-dimensional fractional Brownian motion.

In this paper we study some asymptotic properties of the kernel conditional quantile estimator with randomly left-truncated data which exhibit some kind of dependence. We extend the result obtained by Lemdani, Ould-Saïd and Poulin [16] in the iid case. The uniform strong convergence rate of the estimator under strong mixing hypothesis is obtained.

There has been an explosion of interest in statistical models for analyzing network data, and considerable interest in the class of exponential random graph (ERG) models, especially in connection with difficulties in computing maximum likelihood estimates. The issues associated with these difficulties relate to the broader structure of discrete exponential families. This paper re-examines the issues in two parts. First we consider the closure of k-dimensional exponential families of distribution with discrete base measure and polyhedral convex support P. We show that the normal fan of P is a geometric object that plays a fundamental role in deriving the statistical and geometric properties of the corresponding extended exponential families. We discuss its relevance to maximum likelihood estimation, both from a theoretical and computational standpoint. Second, we apply our results to the analysis of ERG models. By means of a detailed example, we provide some characterization of the properties of ERG models, and, in particular, of certain behaviors of ERG models known as degeneracy.

In this paper, we propose a new lasso-type estimator for censored data after one-step imputatation. While several penalized likelihood estimators have been proposed for censored data variable selection through hazards regression, many such estimators require parametric or proportional hazards assumptions. The proposed estimator, on the other hand, is based on the linear model and least-squares principles. Iterative penalized Buckley-James estimators are also popular in this setting but have been shown to be unstable and unreliable. Unlike path-based learning based on least-squares approximation, our method requires no covariance assumption and the method is valid for even modest sample sizes. Our calibration estimator is the minimizer of a convex loss function using synthetic data and yields reproducible coefficient estimates and coefficient paths. The numerical algorithms are fast, reliable, and readily available because they build on existing software for complete, uncensored data. We examine the large and small sample properties of our estimator and illustrate the method through simulation studies and application to two real data sets.

We present a systematic study of several recently proposed methods of mean field inference for the Dirichlet process mixture (DPM) model. These methods provide approximations to the posterior distribution and are derived using the truncated stick-breaking representation and related approaches. We investigate their use in density estimation and cluster allocation and compare to Monte-Carlo results. Further, more specific topics include the general mathematical structure of the mean field approximation, the handling of the truncation level, the effect of including a prior on the concentration parameter α of the DPM model, the relationship between the proposed variants of the mean field approximation, and the connection to maximum a-posteriori estimation of the DPM model.

Let M_n^(k) denote the kth largest maximum of a sample (X₁,X₂,…,X_n) from parent X with continuous distribution. Assume there exist normalizing constants a_n>0, b_n∈ℝ and a nondegenerate distribution G such that $a_{n}^{-1}(M_{n}^{(1)}-b_{n})\stackrel{w}{\to}G$. Then for fixed k∈ℕ, the almost sure convergence of $$\begin{eqnarray*}\frac{1}{D_{N}}\sum_{n=k}^{N}d_{n}\mathbb{I}\{M_{n}^{(1)}\,{\le}\,a_{n}x_{1}\,{+}\,b_{n},M_{n}^{(2)}\,{\le}\,a_{n}x_{2}\,{+}\,b_{n},\ldots,M_{n}^{(k)}\le a_{n}x_{k}\,{+}\,b_{n}\}\end{eqnarray*}$$ is derived if the positive weight sequence (d_n) with D_N=∑_n=1^Nd_n satisfies conditions provided by Hörmann. Some practical issues of this result are also discussed.

In this paper, a new family of resampling-based penalization procedures for model selection is defined in a general framework. It generalizes several methods, including Efron’s bootstrap penalization and the leave-one-out penalization recently proposed by Arlot (2008), to any exchangeable weighted bootstrap resampling scheme. In the heteroscedastic regression framework, assuming the models to have a particular structure, these resampling penalties are proved to satisfy a non-asymptotic oracle inequality with leading constant close to 1. In particular, they are asympotically optimal. Resampling penalties are used for defining an estimator adapting simultaneously to the smoothness of the regression function and to the heteroscedasticity of the noise. This is remarkable because resampling penalties are general-purpose devices, which have not been built specifically to handle heteroscedastic data. Hence, resampling penalties naturally adapt to heteroscedasticity. A simulation study shows that resampling penalties improve on V-fold cross-validation in terms of final prediction error, in particular when the signal-to-noise ratio is not large.

One of the most popular class of tests for independence between two random variables is the general class of rank statistics which are invariant under permutations. This class contains Spearman’s coefficient of rank correlation statistic, Fisher-Yates statistic, weighted Mann statistic and others. Under the null hypothesis of independence these test statistics have a permutation distribution that is usually approximated by using asymptotic normal theory to determine p-values for these tests. In this note we suggest using a saddlepoint approach that is almost exact and needs no simulations in order to calculate the p-value for tests in this class.

For multivariate normal models and some exponential family models a multiple testing stepwise method is offered that is both admissible and consistent. The method is readily adaptable to selecting variables in linear regression models where it is akin to the forward selection method plus a screening stage plus a sign compatibility stage.

The false discovery rate (FDR) and false nondiscovery rate (FNDR) have received considerable attention in the literature on multiple testing. These performance measures are also appropriate for classification, and in this work we develop generalization error analyses for FDR and FNDR when learning a classifier from labeled training data. Unlike more conventional classification performance measures, the empirical FDR and FNDR are not binomial random variables but rather a ratio of binomials, which introduces challenges not present in conventional formulations of the classification problem. We develop distribution-free uniform deviation bounds and apply these to obtain finite sample bounds and strong universal consistency. We also present a simulation study demonstrating the merits of variance-based bounds, which we also develop. In the context of multiple testing with FDR/FNDR, our framework may be viewed as a way to leverage training data to achieve distribution free, asymptotically optimal inference under the random effects model.

How to weigh the Benjamini-Hochberg procedure? In the context of multiple hypothesis testing, we propose a new step-wise procedure that controls the false discovery rate (FDR) and we prove it to be more powerful than any weighted Benjamini-Hochberg procedure. Both finite-sample and asymptotic results are presented. Moreover, we illustrate good performance of our procedure in simulations and a genomics application. This work is particularly useful in the case of heterogeneous p-value distributions.

The paper focuses on general properties of parametric minimum contrast estimators. The quality of estimation is measured in terms of the rate function related to the contrast, thus allowing to derive exponential risk bounds invariant with respect to the detailed probabilistic structure of the model. This approach works well for small or moderate samples and covers the case of a misspecified parametric model. Another important feature of the presented bounds is that they may be used in the case when the parametric set is not compact. These bounds do not rely on the entropy or covering numbers and can be easily computed. The most important statistical fact resulting from the exponential bonds is a concentration inequality which claims that minimum contrast estimators concentrate with a large probability on the level set of the rate function. In typical situations, every such set is a root-n neighborhood of the parameter of interest. We also show that the obtained bounds can help for bounding the estimation risk and constructing confidence sets for the underlying parameters. Our general results are illustrated for the case of an i.i.d. sample. We also consider several popular examples including least absolute deviation estimation and the problem of estimating the location of a change point. What we obtain in these examples slightly differs from the usual asymptotic results presented in statistical literature. This difference is due to the unboundness of the parameter set and a possible model misspecification.

We consider the standard non-parametric regression model with Gaussian errors but where the data consist of different samples. The question to be answered is whether the samples can be adequately represented by the same regression function. To do this we define for each sample a universal, honest and non-asymptotic confidence region for the regression function. Any subset of the samples can be represented by the same function if and only if the intersection of the corresponding confidence regions is non-empty. If the empirical supports of the samples are disjoint then the intersection of the confidence regions is always non–empty and a negative answer can only be obtained by placing shape or quantitative smoothness conditions on the joint approximation, or by making additional assumptions about the support points. Alternatively, a simplest joint approximation function can be calculated which gives a measure of the cost of the joint approximation, for example, the number of extra peaks required.

Here we propose a penalized orthogonal-components regression (POCRE) for large p small n data. Orthogonal components are sequentially constructed to maximize, upon standardization, their correlation to the response residuals. A new penalization framework, implemented via empirical Bayes thresholding, is presented to effectively identify sparse predictors of each component. POCRE is computationally efficient owing to its sequential construction of leading sparse principal components. In addition, such construction offers other properties such as grouping highly correlated predictors and allowing for collinear or nearly collinear predictors. With multivariate responses, POCRE can construct common components and thus build up latent-variable models for large p small n data.

Two areas of research – longitudinal data analysis and kernel machines – have large, but mostly distinct, literatures. This article shows explicitly that both fields have much in common with each other. In particular, many popular longitudinal data fitting procedures are special types of kernel machines. These connections have the potential to provide fruitful cross-fertilization between longitudinal data analytic and kernel machine methodology.

Kernel methods are closely related to the notion of reproducing kernel Hilbert space (RKHS). A kernel machine is based on the minimization of an empirical cost and a stabilizer (usually the norm in the RKHS). In this paper we propose to use Besov spaces as alternative hypothesis spaces. We study statistical performances of a penalized empirical risk minimization for classification where the stabilizer is a Besov norm. More precisely, we state fast rates of convergence to the Bayes rule. These rates are adaptive with respect to the regularity of the Bayes.

Let {X_k,k⩾1} be a stationary Gaussian sequence with partial maximum M_n=max {X_k,1⩽k⩽n} and sample mean X̅_n=∑_k=1ⁿX_k/n. Suppose that some of the random variables X₁,X₂,… can be observed and the others not. Denote by M̃_n the maximum of the observed random variables from the set {X₁,X₂,…,X_n}. Under some mild conditions, we prove the joint limiting distribution and the almost sure limit theorem for (M̃_n−X̅_n,M_n−X̅_n).

The problem of prediction in functional linear regression is conventionally addressed by reducing dimension via the standard principal component basis. In this paper we show that weighted least-squares methods can be more effective when the experimental errors are heteroscedastic. We give a concise theoretical result which demonstrates the effectiveness of this approach, even when the model for the variance is inaccurate, and we explore the numerical properties of the method. We show too that the advantages of the suggested adaptive techniques are not found only in low-dimensional aspects of the problem; rather, they accrue almost equally among all dimensions.

We study the problem of fitting circles (or circular arcs) to data points observed with errors in both variables. A detailed error analysis for all popular circle fitting methods – geometric fit, Kåsa fit, Pratt fit, and Taubin fit – is presented. Our error analysis goes deeper than the traditional expansion to the leading order. We obtain higher order terms, which show exactly why and by how much circle fits differ from each other. Our analysis allows us to construct a new algebraic (non-iterative) circle fitting algorithm that outperforms all the existing methods, including the (previously regarded as unbeatable) geometric fit.

Consider an R×C contingency table in which the categories are ordered. Multiple testing of the hypotheses that each local odds ratio is one is carried out. The methodology to perform the multiple tests is an extension of the MRDSS method of Chen, Cohen, and Sackrowitz (2009). The MRDSS method extends the MRD method of Cohen, Sackrowitz, and Xu (2009) by adding a screen stage and a sign stage to MRD. The MRDSS method as well as the extension here is admissible and consistent. Both Fisher-type statistics and Chi-square statistics are used. Examples and a simulation study are included.

We introduce and study distributions of sets of binary variables that are symmetric, that is each has equally probable levels. The joint distribution of these special types of binary variables, if generated by a recursive process of linear main effects is essentially parametrized in terms of marginal correlations. This contrasts with the log-linear formulation of joint probabilities in which parameters measure conditional associations given all remaining variables. The new formulation permits useful comparisons of different types of graphical Markov models and leads to a close approximation of Gaussian orthant probabilities.

This paper studies the quasi-maximum likelihood estimator (QMLE) of non-stationary GARCH(p,q) models. By expressing GARCH models in matrix form, the log-likelihood function is written in terms of the product of random matrices. Oseledec’s multiplicative ergodic theorem is then used to establish the asymptotic properties of the log-likelihood function and thereby, showing the weak consistency and the asymptotic normality of the QMLE for non-stationary GARCH(p,q) models.

In extreme value statistics for stationary sequences, blocks estimators are usually constructed by using disjoint blocks because exceedances over high thresholds of different blocks can be assumed asymptotically independent. In this paper we focus on the estimation of the extremal index which measures the degree of clustering of extremes. We consider disjoint and sliding blocks estimators and compare their asymptotic properties. In particular we show that the sliding blocks estimator is more efficient than the disjoint version and has a smaller asymptotic bias. Moreover we propose a method to reduce its bias when considering sufficiently large block sizes.

Methods are proposed to construct empirical measures when there are missing terms among the components of a random vector. Furthermore, Vapnik-Chevonenkis type exponential bounds are obtained on the uniform deviations of these estimators, from the true probabilities. These results can then be used to deal with classical problems such as statistical classification, via empirical risk minimization, when there are missing covariates among the data. Another application involves the uniform estimation of a distribution function.

Much is now known about the consistency of Bayesian updating on infinite-dimensional parameter spaces with independent or Markovian data. Necessary conditions for consistency include the prior putting enough weight on the correct neighborhoods of the data-generating distribution; various sufficient conditions further restrict the prior in ways analogous to capacity control in frequentist nonparametrics. The asymptotics of Bayesian updating with mis-specified models or priors, or non-Markovian data, are far less well explored. Here I establish sufficient conditions for posterior convergence when all hypotheses are wrong, and the data have complex dependencies. The main dynamical assumption is the asymptotic equipartition (Shannon-McMillan-Breiman) property of information theory. This, along with Egorov’s Theorem on uniform convergence, lets me build a sieve-like structure for the prior. The main statistical assumption, also a form of capacity control, concerns the compatibility of the prior and the data-generating process, controlling the fluctuations in the log-likelihood when averaged over the sieve-like sets. In addition to posterior convergence, I derive a kind of large deviations principle for the posterior measure, extending in some cases to rates of convergence, and discuss the advantages of predicting using a combination of models known to be wrong. An appendix sketches connections between these results and the replicator dynamics of evolutionary theory.

Inspired by the circle fitting algorithm “Hyper” of Al-Sharadqah and Chernov [1], which eliminates the second order “essential bias” that excludes terms of O(σ²/N²), we extend their analysis and show that by a small modification the second order bias can be eliminated completely. By numerical experiments, we show that this results in better performance when the number N of points is small and the noise is large.

In this paper we deal with the regression problem in a random design setting. We investigate asymptotic optimality under minimax point of view of various Bayesian rules based on warped wavelets. We show that they nearly attain optimal minimax rates of convergence over the Besov smoothness class considered. Warped wavelets have been introduced recently, they offer very good computable and easy-to-implement properties while being well adapted to the statistical problem at hand. We particularly put emphasis on Bayesian rules leaning on small and large variance Gaussian priors and discuss their simulation performances, comparing them with a hard thresholding procedure.

We discuss inference for multiple binary sequences under the hypothesis of Markov exchangeability. So far, the only kind of models for this purpose have been the mixtures of Markov chains. We present a new class of hierarchical models parameterized in terms of Bahadur/Lancaster’s interactions, and compare it to the mixtures of Markov chains models.

We present a graph-based technique for estimating sparse covariance matrices and their inverses from high-dimensional data. The method is based on learning a directed acyclic graph (DAG) and estimating parameters of a multivariate Gaussian distribution based on a DAG. For inferring the underlying DAG we use the PC-algorithm [27] and for estimating the DAG-based covariance matrix and its inverse, we use a Cholesky decomposition approach which provides a positive (semi-)definite sparse estimate. We present a consistency result in the high-dimensional framework and we compare our method with the Glasso [12, 8, 2] for simulated and real data.

The empirical likelihood method is a versatile approach for testing hypotheses and constructing confidence regions in a non-parametric setting. For testing the value of a vector mean, the empirical likelihood method offers the benefit of making no distributional assumptions beyond some mild moment conditions. However, in small samples or high dimensions the method is very poorly calibrated, producing tests that generally have a much higher type I error than the nominal level, and it suffers from a limiting convex hull constraint. Methods to address the performance of the empirical likelihood in the vector mean setting have been proposed in a number of papers, including a contribution that suggests supplementing the observed dataset with an artificial data point. We examine the consequences of this approach and describe a limitation of their method that we have discovered in settings when the sample size is relatively small compared with the dimension. We propose a new modification to the extra data approach that involves adding two points and changing the location of the extra points. We explore the benefits that this modification offers, and show that it results in better calibration, particularly in difficult cases. This new approach also results in a small-sample connection between the modified empirical likelihood method and Hotelling’s T-square test. We show that varying the location of the added data points creates a continuum of tests that range from the unmodified empirical likelihood statistic to Hotelling’s T-square statistic.

Semiparametric regression is a fusion between parametric regression and nonparametric regression that integrates low-rank penalized splines, mixed model and hierarchical Bayesian methodology – thus allowing more streamlined handling of longitudinal and spatial correlation. We review progress in the field over the five-year period between 2003 and 2007. We find semiparametric regression to be a vibrant field with substantial involvement and activity, continual enhancement and widespread application.

Dimension reduction and variable selection are performed routinely in case-control studies, but the literature on the theoretical aspects of the resulting estimates is scarce. We bring our contribution to this literature by studying estimators obtained via ℓ₁ penalized likelihood optimization. We show that the optimizers of the ℓ₁ penalized retrospective likelihood coincide with the optimizers of the ℓ₁ penalized prospective likelihood. This extends the results of Prentice and Pyke (1979), obtained for non-regularized likelihoods. We establish both the sup-norm consistency of the odds ratio, after model selection, and the consistency of subset selection of our estimators. The novelty of our theoretical results consists in the study of these properties under the case-control sampling scheme. Our results hold for selection performed over a large collection of candidate variables, with cardinality allowed to depend and be greater than the sample size. We complement our theoretical results with a novel approach of determining data driven tuning parameters, based on the bisection method. The resulting procedure offers significant computational savings when compared with grid search based methods. All our numerical experiments support strongly our theoretical findings.

Variable selection for high-dimensional linear models has received a lot of attention lately, mostly in the context of ℓ₁-regularization. Part of the attraction is the variable selection effect: parsimonious models are obtained, which are very suitable for interpretation. In terms of predictive power, however, these regularized linear models are often slightly inferior to machine learning procedures like tree ensembles. Tree ensembles, on the other hand, lack usually a formal way of variable selection and are difficult to visualize. A Garrote-style convex penalty for trees ensembles, in particular Random Forests, is proposed. The penalty selects functional groups of nodes in the trees. These could be as simple as monotone functions of individual predictor variables. This yields a parsimonious function fit, which lends itself easily to visualization and interpretation. The predictive power is maintained at least at the same level as the original tree ensemble. A key feature of the method is that, once a tree ensemble is fitted, no further tuning parameter needs to be selected. The empirical performance is demonstrated on a wide array of datasets.

We consider the minimax rate of testing or estimation of nonlinear functionals defined on semiparametric models. Existing methods appear not capable of determining a lower bound on the minimax rate if the semiparametric model is indexed by several infinite-dimensional parameters. These methods test a single null distribution to a convex mixture of perturbed distributions. To cope with semiparametric functionals we extend these methods to comparing two convex mixtures. The first mixture is obtained by perturbing a first parameter of the model, and the second by perturbing in addition a second parameter. We obtain a lower bound on the affinity of the resulting pair of mixtures of product measures in terms of three parameters that measure the sizes and asymmetry of the perturbations. We apply the new result to two examples: the estimation of a mean response when response data are missing at random, and the estimation of an expected conditional covariance.

We consider estimation of a step function f from noisy observations of a deconvolution ϕ*f, where ϕ is some bounded L₁-function. We use a penalized least squares estimator to reconstruct the signal f from the observations, with penalty equal to the number of jumps of the reconstruction. Asymptotically, it is possible to correctly estimate the number of jumps with probability one. Given that the number of jumps is correctly estimated, we show that for a bounded kernel ϕ the corresponding estimates of the jump locations and jump heights are n^−1/2 consistent and converge to a joint normal distribution with covariance structure depending on ϕ. As special case we obtain the asymptotic distribution of the least squares estimator in multiphase regression and generalizations thereof. Finally, singular integral kernels are briefly discussed and it is shown that the n^−1/2-rate can be improved.

Oracle inequalities and variable selection properties for the Lasso in linear models have been established under a variety of different assumptions on the design matrix. We show in this paper how the different conditions and concepts relate to each other. The restricted eigenvalue condition [2] or the slightly weaker compatibility condition [18] are sufficient for oracle results. We argue that both these conditions allow for a fairly general class of design matrices. Hence, optimality of the Lasso for prediction and estimation holds for more general situations than what it appears from coherence [5, 4] or restricted isometry [10] assumptions.

The Rosenblatt process is a self-similar non-Gaussian process which lives in second Wiener chaos, and occurs as the limit of correlated random sequences in so-called “non-central limit theorems”. It shares the same covariance as fractional Brownian motion. We study the asymptotic distribution of the quadratic variations of the Rosenblatt process based on long filters, including filters based on high-order finite-difference and wavelet-based schemes. We find exact formulas for the limiting distributions, which we then use to devise strongly consistent estimators of the self-similarity parameter H. Unlike the case of fractional Brownian motion, no matter now high the filter orders are, the estimators are never asymptotically normal, converging instead in the mean square to the observed value of the Rosenblatt process at time 1.

A reduced-bias nonparametric estimator of the cumulative distribution function (CDF) and the survival function is proposed using infinite-order kernels. Fourier transform theory on generalized functions is utilized to obtain the improved bias estimates. The new estimators are analyzed in terms of their relative deficiency to the empirical distribution function and Kaplan-Meier estimator, and even improvements in terms of asymptotic relative efficiency (ARE) are present under specified assumptions on the data. The deficiency analysis introduces a deficiency rate which provides a continuum between the classical deficiency analysis and an efficiency analysis. Additionally, an automatic bandwidth selection algorithm, specially tailored to the infinite-order kernels, is incorporated into the estimators. In small sample sizes these estimators can significantly improve the estimation of the CDF and survival function as is illustrated through the deficiency analysis and computer simulations.

Here we explore general asymptotic properties of Predictive Recursion (PR) for nonparametric estimation of mixing distributions. We prove that, when the mixture model is mis-specified, the estimated mixture converges almost surely in total variation to the mixture that minimizes the Kullback-Leibler divergence, and a bound on the (Hellinger contrast) rate of convergence is obtained. Simulations suggest that this rate is nearly sharp in a minimax sense. Moreover, when the model is identifiable, almost sure weak convergence of the mixing distribution estimate follows.

PR assumes that the support of the mixing distribution is known. To remove this requirement, we propose a generalization that incorporates a sequence of supports, increasing with the sample size, that combines the efficiency of PR with the flexibility of mixture sieves. Under mild conditions, we obtain a bound on the rate of convergence of these new estimates.

Clustering is one of the most useful tools for high-dimensional analysis, e.g., for microarray data. It becomes challenging in presence of a large number of noise variables, which may mask underlying clustering structures. Therefore, noise removal through variable selection is necessary. One effective way is regularization for simultaneous parameter estimation and variable selection in model-based clustering. However, existing methods focus on regularizing the mean parameters representing centers of clusters, ignoring dependencies among variables within clusters, leading to incorrect orientations or shapes of the resulting clusters. In this article, we propose a regularized Gaussian mixture model with general covariance matrices, taking various dependencies into account. At the same time, this approach shrinks the means and covariance matrices, achieving better clustering and variable selection. To overcome one technical challenge in estimating possibly large covariance matrices, we derive an E-M algorithm to utilize the graphical lasso (Friedman et al. 2007) for parameter estimation. Numerical examples, including applications to microarray gene expression data, demonstrate the utility of the proposed method.

We consider the statistical analysis of random sections of a spin fibre bundleover the sphere. These may be thought of as random fields that at each point $p\in \mathbb{S}^{2}$ take as a value a curve (e.g. an ellipse) living in the tangent plane at that point $T_{p}\mathbb{S}^{2}$, rather than a number as in ordinary situations. The analysis of such fields is strongly motivated by applications, for instance polarization experiments in Cosmology. To investigate such fields, spin needlets were recently introduced by [21] and [20]. We consider the use of spin needlets for spin angular power spectrum estimation, in the presence of noise and missing observations, and we provide Central Limit Theorem results, in the high frequency sense; we discuss also tests for bias and asymmetries with an asymptotic justification.

Physical random numbers are not as widely used in Monte Carlo integration as pseudo-random numbers are. They are inconvenient for many reasons. If we want to generate them on the fly, then they may be slow. When we want reproducible results from them, we need a lot of storage. This paper shows that we may construct N=n(n−1)/2 pairwise independent random vectors from n independent ones, by summing them modulo 1 in pairs. As a consequence, the storage and speed problems of physical random numbers can be greatly mitigated. The new vectors lead to Monte Carlo averages with the same mean and variance as if we had used N independent vectors. The asymptotic distribution of the sample mean has a surprising feature: it is always symmetric, but never Gaussian. This follows by writing the sample mean as a degenerate U-statistic whose kernel is a left-circulant matrix. Because of the symmetry, a small number B of replicates can be used to get confidence intervals based on the central limit theorem.

In an influential work, Qin and Lawless (1994) proposed a general estimating equations (GEE) formulation for maximum empirical likelihood (MEL) estimation and inference. The formulation replaces a model specified by GEE with a set of data-supported probability mass functions that satisfy empirical estimating equations (E³). In this paper we use several examples from the literature to demonstrate that the set may be empty for some E³ models and finite data samples. As a result, MEL does not exist for such models and data sets. If MEL and other E³-based methods are to be used, then models will have to be checked on case-by-case basis for the absence or presence of the empty set problem.

We define and investigate a new class of measure-valued Markov chains by resorting to ideas formulated in Bayesian nonparametrics related to the Dirichlet process and the Gibbs sampler. Dependent random probability measures in this class are shown to be stationary and ergodic with respect to the law of a Dirichlet process and to converge in distribution to the neutral diffusion model.

We study and compare three estimators of a discrete monotone distribution: (a) the (raw) empirical estimator; (b) the “method of rearrangements” estimator; and (c) the maximum likelihood estimator. We show that the maximum likelihood estimator strictly dominates both the rearrangement and empirical estimators in cases when the distribution has intervals of constancy. For example, when the distribution is uniform on {0,…,y}, the asymptotic risk of the method of rearrangements estimator (in squared ℓ₂ norm) is y/(y+1), while the asymptotic risk of the MLE is of order (logy)/(y+1). For strictly decreasing distributions, the estimators are asymptotically equivalent.