A popular model selection approach for generalized linear mixed-effects models is the Akaike information criterion, or AIC. Among others, [7] pointed out the distinction between the marginal and conditional inference depending on the focus of research. The conditional AIC was derived for the linear mixed-effects model which was later generalized by [5]. We show that the similar strategy extends to Poisson regression with random effects, where conditional AIC can be obtained based on our observations. Simulation studies demonstrate the usage of the criterion.

Many applications in the field of statistics require Markov chain Monte Carlo methods. Determining appropriate starting values and run lengths can be both analytically and empirically challenging. A desire to overcome these problems has led to the development of exact, or perfect, sampling algorithms which convert a Markov chain into an algorithm that produces i.i.d. samples from the stationary distribution. Unfortunately, very few of these algorithms have been developed for the distributions that arise in statistical applications, which typically have uncountable support. Here we study an exact sampling algorithm using a geometrically ergodic Markov chain on a general state space. Our work provides a significant reduction to the number of input draws necessary for the Bernoulli factory, which enables exact sampling via a rejection sampling approach. We illustrate the algorithm on a univariate Metropolis-Hastings sampler and a bivariate Gibbs sampler, which provide a proof of concept and insight into hyper-parameter selection. Finally, we illustrate the algorithm on a Bayesian version of the one-way random effects model with data from a styrene exposure study.

Consider the standard Gaussian linear regression model Y=Xθ₀+ε, where Y∈ℝⁿ is a response vector and X∈ℝ^n×p is a design matrix. Numerous work have been devoted to building efficient estimators of θ₀ when p is much larger than n. In such a situation, a classical approach amounts to assume that θ₀ is approximately sparse. This paper studies the minimax risks of estimation and testing over classes of k-sparse vectors θ₀. These bounds shed light on the limitations due to high-dimensionality. The results encompass the problem of prediction (estimation of Xθ₀), the inverse problem (estimation of θ₀) and linear testing (testing Xθ₀=0). Interestingly, an elbow effect occurs when the number of variables klog(p/k) becomes large compared to n. Indeed, the minimax risks and hypothesis separation distances blow up in this ultra-high dimensional setting. We also prove that even dimension reduction techniques cannot provide satisfying results in an ultra-high dimensional setting. Moreover, we compute the minimax risks when the variance of the noise is unknown. The knowledge of this variance is shown to play a significant role in the optimal rates of estimation and testing. All these minimax bounds provide a characterization of statistical problems that are so difficult so that no procedure can provide satisfying results.

The aim of this paper is to establish non-asymptotic minimax rates for goodness-of-fit hypotheses testing in an heteroscedastic setting. More precisely, we deal with sequences (Y_j)_j∈J of independent Gaussian random variables, having mean (θ_j)_j∈J and variance (σ_j)_j∈J. The set J will be either finite or countable. In particular, such a model covers the inverse problem setting where few results in test theory have been obtained. The rates of testing are obtained with respect to l₂ norm, without assumption on (σ_j)_j∈J and on several functions spaces. Our point of view is entirely non-asymptotic.

Exponential families are often used to model data sets with complex dependence. Maximum likelihood estimators (MLE) can be difficult to estimate when the likelihood is expensive to compute. Markov chain Monte Carlo (MCMC) methods based on the MCMC-MLE algorithm in [17] are guaranteed to converge in theory under certain conditions when starting from any value, but in practice such an algorithm may labor to converge when given a poor starting value. We present a simple line search algorithm to find the MLE of a regular exponential family when the MLE exists and is unique. The algorithm can be started from any initial value and avoids the trial and error experimentation associated with calibrating algorithms like stochastic approximation. Unlike many optimization algorithms, this approach utilizes first derivative information only, evaluating neither the likelihood function itself nor derivatives of higher order than first. We show convergence of the algorithm for the case where the gradient can be calculated exactly. When it cannot, it has a particularly convenient form that is easily estimable with MCMC, making the algorithm still useful to a practitioner.

K-means clustering is a widely used tool for cluster analysis due to its conceptual simplicity and computational efficiency. However, its performance can be distorted when clustering high-dimensional data where the number of variables becomes relatively large and many of them may contain no information about the clustering structure. This article proposes a high-dimensional cluster analysis method via regularized k-means clustering, which can simultaneously cluster similar observations and eliminate redundant variables. The key idea is to formulate the k-means clustering in a form of regularization, with an adaptive group lasso penalty term on cluster centers. In order to optimally balance the trade-off between the clustering model fitting and sparsity, a selection criterion based on clustering stability is developed. The asymptotic estimation and selection consistency of the regularized k-means clustering with diverging dimension is established. The effectiveness of the regularized k-means clustering is also demonstrated through a variety of numerical experiments as well as applications to two gene microarray examples. The regularized clustering framework can also be extended to the general model-based clustering.

This paper proposes a new algorithm for Bayesian model determination in Gaussian graphical models under G-Wishart prior distributions. We first review recent development in sampling from G-Wishart distributions for given graphs, with a particular interest in the efficiency of the block Gibbs samplers and other competing methods. We generalize the maximum clique block Gibbs samplers to a class of flexible block Gibbs samplers and prove its convergence. This class of block Gibbs samplers substantially outperforms its competitors along a variety of dimensions. We next develop the theory and computational details of a novel Markov chain Monte Carlo sampling scheme for Gaussian graphical model determination. Our method relies on the partial analytic structure of G-Wishart distributions integrated with the exchange algorithm. Unlike existing methods, the new method requires neither proposal tuning nor evaluation of normalizing constants of G-Wishart distributions.

Representing distributions over permutations can be a daunting task due to the fact that the number of permutations of n objects scales factorially in n. One recent way that has been used to reduce storage complexity has been to exploit probabilistic independence, but as we argue, full independence assumptions impose strong sparsity constraints on distributions and are unsuitable for modeling rankings. We identify a novel class of independence structures, called riffled independence, encompassing a more expressive family of distributions while retaining many of the properties necessary for performing efficient inference and reducing sample complexity. In riffled independence, one draws two permutations independently, then performs the riffle shuffle, common in card games, to combine the two permutations to form a single permutation. Within the context of ranking, riffled independence corresponds to ranking disjoint sets of objects independently, then interleaving those rankings. In this paper, we provide a formal introduction to riffled independence and propose an automated method for discovering sets of items which are riffle independent from a training set of rankings. We show that our clustering-like algorithms can be used to discover meaningful latent coalitions from real preference ranking datasets and to learn the structure of hierarchically decomposable models based on riffled independence.

In this paper we are concerned with fully automatic and locally adaptive estimation of functions in a “signal + noise”-model where the regression function may additionally be blurred by a linear operator, e.g. by a convolution. To this end, we introduce a general class of statistical multiresolution estimators and develop an algorithmic framework for computing those. By this we mean estimators that are defined as solutions of convex optimization problems with ℓ_∞-type constraints. We employ a combination of the alternating direction method of multipliers with Dykstra’s algorithm for computing orthogonal projections onto intersections of convex sets and prove numerical convergence. The capability of the proposed method is illustrated by various examples from imaging and signal detection.

We present two sets of theoretical results on the grouped lasso with overlap due to Jacob, Obozinski and Vert (2009) in the linear regression setting. This method jointly selects predictors in sparse regression, allowing for complex structured sparsity over the predictors encoded as a set of groups. This flexible framework suggests that arbitrarily complex structures can be encoded with an intricate set of groups. Our results show that this strategy results in unexpected theoretical consequences for the procedure. In particular, we give two sets of results: (1) finite sample bounds on prediction and estimation, and (2) asymptotic distribution and selection. Both sets of results demonstrate negative consequences from choosing an increasingly complex set of groups for the procedure, as well for when the set of groups cannot recover the true sparsity pattern. Additionally, these results demonstrate the differences and similarities between the the grouped lasso procedure with and without overlapping groups. Our analysis shows that while the procedure enjoys advantages over the standard lasso, the set of groups must be chosen with caution — an overly complex set of groups will damage the analysis.

An empirical likelihood (EL) estimator was proposed by Qin and Zhang (2007) for improving the inverse probability weighting estimation in a missing response problem. The authors showed by simulation studies that the finite sample performance of EL estimator is better than certain existing estimators and they also showed large sample results for the estimator. However, the empirical likelihood estimator does not have a uniformly smaller asymptotic variance than other existing estimators in general. We consider several modifications to the empirical likelihood estimator and show that the proposed estimator dominates the empirical likelihood estimator and several other existing estimators in terms of asymptotic efficiencies under missing at random. The proposed estimator also attains the minimum asymptotic variance among estimators having influence functions in a certain class and enjoys certain double robustness properties.

In this paper, in order to deal with data rounding issues, we introduce a new class of estimators of the Hurst exponent of the fractional Brownian motion (fBm) process. These estimators are based on sample expectiles of discrete variations of a sample path of the fBm process. So as to derive the statistical properties of the proposed estimators, we establish asymptotic results for sample expectiles of subordinated stationary Gaussian processes with unit variance and correlation function satisfying ρ(i)∼κ|i|^−α (κ∈ℝ) with α>0. Via a simulation study, we demonstrate the relevance of the expectile-based estimation method and show that the suggested estimators are more robust to data rounding than their sample quantile-based counterparts.

One of the challenges with functional data is incorporating geometric structure, or local correlation, into the analysis. This structure is inherent in the output from an increasing number of biomedical technologies, and a functional linear model is often used to estimate the relationship between the predictor functions and scalar responses. Common approaches to the problem of estimating a coefficient function typically involve two stages: regularization and estimation. Regularization is usually done via dimension reduction, projecting onto a predefined span of basis functions or a reduced set of eigenvectors (principal components). In contrast, we present a unified approach that directly incorporates geometric structure into the estimation process by exploiting the joint eigenproperties of the predictors and a linear penalty operator. In this sense, the components in the regression are ‘partially empirical’ and the framework is provided by the generalized singular value decomposition (GSVD). The form of the penalized estimation is not new, but the GSVD clarifies the process and informs the choice of penalty by making explicit the joint influence of the penalty and predictors on the bias, variance and performance of the estimated coefficient function. Laboratory spectroscopy data and simulations are used to illustrate the concepts.

We prove that the posterior distribution of a parameter in misspecified LAN parametric models can be approximated by a random normal distribution. We derive from this that Bayesian credible sets are not valid confidence sets if the model is misspecified. We obtain the result under conditions that are comparable to those in the well-specified situation: uniform testability against fixed alternatives and sufficient prior mass in neighbourhoods of the point of convergence. The rate of convergence is considered in detail, with special attention for the existence and construction of suitable test sequences. We also give a lemma to exclude testable model subsets which implies a misspecified version of Schwartz’ consistency theorem, establishing weak convergence of the posterior to a measure degenerate at the point at minimal Kullback-Leibler divergence with respect to the true distribution.

We focus on the supervised binary classification problem, which consists in guessing the label Y associated to a co-variate X∈ℝ^d, given a set of n independent and identically distributed co-variates and associated labels (X_i,Y_i). We assume that the law of the random vector (X,Y) is unknown and the marginal law of X admits a density supported on a set ${\mathcal{A}}$. In the particular case of plug-in classifiers, solving the classification problem boils down to the estimation of the regression function $\eta(X)=\mathbb {E}[Y|X]$. Assuming first ${\mathcal{A}}$ to be known, we show how it is possible to construct an estimator of η by localized projections onto a multi-resolution analysis (MRA). In a second step, we show how this estimation procedure generalizes to the case where ${\mathcal{A}}$ is unknown. Interestingly, this novel estimation procedure presents similar theoretical performances as the celebrated local-polynomial estimator (LPE). In addition, it benefits from the lattice structure of the underlying MRA and thus outperforms the LPE from a computational standpoint, which turns out to be a crucial feature in many practical applications. Finally, we prove that the associated plug-in classifier can reach super-fast rates under a margin assumption.

This paper examines the local power of the likelihood ratio, Wald, score and gradient tests under the presence of a scalar parameter, ϕ say, that is orthogonal to the remaining parameters. We show that some of the coefficients that define the local powers remain unchanged regardless of whether ϕ is known or needs to be estimated, whereas the others can be written as the sum of two terms, the first of which being the corresponding term obtained as if ϕ were known, and the second, an additional term yielded by the fact that ϕ is unknown. The contribution of each set of parameters on the local powers of the tests can then be examined. Various implications of our main result are stated and discussed. Several examples are presented for illustrative purposes.

This paper is devoted to the off-line multiple breaks detection for a general class of models. The observations are supposed to fit a parametric causal process (such as classical models AR(∞), ARCH(∞) or TARCH(∞)) with distinct parameters on multiple periods. The number and dates of breaks, and the different parameters on each period are estimated using a quasi-likelihood contrast penalized by the number of distinct periods. For a convenient choice of the regularization parameter in the penalty term, the consistency of the estimator is proved when the moment order r of the process satisfies r≥2. If r≥4, the length of each approximative segment tends to infinity at the same rate as the length of the true segment and the parameters estimators on each segment are asymptotically normal. Compared to the existing literature, we added the fact that a dependence is possible over distinct periods. To be robust to this dependence, the chosen regularization parameter in the penalty term is larger than the ones from BIC approach. We detail our results which notably improve the existing ones for the AR(∞), ARCH(∞) and TARCH(∞) models. For the practical applications (when n is not too large) we use a data-driven procedure based on the slope estimation to choose the penalty term. The procedure is implemented using the dynamic programming algorithm. It is an O(n²) complexity algorithm that we apply on AR(1), AR(2), GARCH(1,1) and TARCH(1) processes and on the FTSE index data.

This paper studies the problem of local bandwidth selection for local linear regression. It is known that the optimal local bandwidth for estimating the unknown curve f at design point x depends on the curve’s second derivative f''(x) at x. Therefore one could select the local bandwidth h(x) at x via estimating f''(x). However, as typically estimating f''(x) is a much harder task than estimating f(x) itself, this approach for choosing h(x) tends to produce less accurate results. This paper proposes a method for choosing h(x) that bypasses the estimation of f''(x), yet at the same time utilizes the useful fact that the optimal local bandwidth depends on f''(x). The main idea is to first partition the domain of f(x) into different segments for which the second derivative of each segment is approximately constant. The number and the length of the segments are assumed unknown and will be estimated. Then, after such a partition is obtained, any reliable, well-studied global bandwidth selection method can be applied to choose the bandwidth for each segment. The empirical performance of the proposed local bandwidth selection method is evaluated by numerical experiments.

In some applications with astronomical and survival data, doubly truncated data are sometimes encountered. In this work we introduce kernel-type density estimation for a random variable which is sampled under random double truncation. Two different estimators are considered. As usual, the estimators are defined as a convolution between a kernel function and an estimator of the cumulative distribution function, which may be the NPMLE [2] or a semiparametric estimator [9]. Asymptotic properties of the introduced estimators are explored. Their finite sample behaviour is investigated through simulations. Real data illustration is included.

We consider a general high-dimensional additive hazards model in a non-asymptotic setting, including regression for censored-data. In this context, we consider a Lasso estimator with a fully data-driven ℓ₁ penalization, which is tuned for the estimation problem at hand. We prove sharp oracle inequalities for this estimator. Our analysis involves a new “data-driven” Bernstein’s inequality, that is of independent interest, where the predictable variation is replaced by the optional variation.

We consider a situation where the state of a system is represented by a real-valued vector x∈ℝⁿ. Under normal circumstances, the vector x is zero, while an event manifests as non-zero entries in x, possibly few. Our interest is in designing algorithms that can reliably detect events — i.e., test whether x=0 or x≠0 — with the least amount of information. We place ourselves in a situation, now common in the signal processing literature, where information on x comes in the form of noisy linear measurements y=〈a,x〉+z, where a∈ℝⁿ has norm bounded by 1 and $z\in \mathcal{N}(0,1)$ We derive information bounds in an active learning setup and exhibit some simple near-optimal algorithms. In particular, our results show that the task of detection within this setting is at once much easier, simpler and different than the tasks of estimation and support recovery.

Recent technological advances provide researchers with a way of gathering real-time information on an individual’s movement through the use of wearable devices that record acceleration. In this paper, we propose a method for identifying activity types, like walking, standing, and resting, from acceleration data. Our approach decomposes movements into short components called “movelets”, and builds a reference for each activity type. Unknown activities are predicted by matching new movelets to the reference. We apply our method to data collected from a single, three-axis accelerometer and focus on activities of interest in studying physical function in elderly populations. An important technical advantage of our methods is that they allow identification of short activities, such as taking two or three steps and then stopping, as well as low frequency rare(compared with the whole time series) activities, such as sitting on a chair. Based on our results we provide simple and actionable recommendations for the design and implementation of large epidemiological studies that could collect accelerometry data for the purpose of predicting the time series of activities and connecting it to health outcomes.

We consider the estimation of a bounded regression function with nonparametric heteroscedastic noise and random design. We study the true and empirical excess risks of the least-squares estimator on finite-dimensional vector spaces. We give upper and lower bounds on these quantities that are nonasymptotic and optimal to first order, allowing the dimension to depend on sample size. These bounds show the equivalence between the true and empirical excess risks when, among other things, the least-squares estimator is consistent in sup-norm with the projection of the regression function onto the considered model. Consistency in the sup-norm is then proved for suitable histogram models and more general models of piecewise polynomials that are endowed with a localized basis structure.

Asymptotic properties of the generalized information criterion for model selection are examined and new conditions under which this criterion is overfitting, consistent, or underfitting are derived.

For a locally smooth statistical model, we investigate kernel U-quantiles estimators. Under suitable assumptions, we establish a strong Bahadur representation theorem, an invariance principle, and the asymptotic normality for randomly indexed sequences of observations.

Consider a case-control study in which the aim is to assess a factor’s effect on disease occurrence. We suppose that this factor is dichotomous. Also suppose that the data consists of two strata, each stratum summarized by a two-by-two table. A commonly-proposed two-stage analysis of this type of data is the following. We carry out a preliminary test of homogeneity of the stratum-specific odds ratios. If the null hypothesis of homogeneity is accepted then we find a confidence interval for the assumed common value (across strata) of the odds ratio. We examine the statistical properties of this two-stage analysis, based on the Woolf method, on confidence intervals constructed for the stratum-specific odds ratios, for large numbers of cases and controls for each stratum. We provide both a Monte Carlo simulation method and an elegant large-sample method for this examination. These methods are applied to obtain numerical results in the context of the large numbers of cases and controls for each stratum that arose in a real-life dataset. In this context, we find that the preliminary test of homogeneity of the stratum-specific odds ratios has a very harmful effect on the coverage probabilities of these confidence intervals.

In statistical practice, whether a Bayesian or frequentist approach is used in inference depends not only on the availability of prior information but also on the attitude taken toward partial prior information, with frequentists tending to be more cautious than Bayesians. The proposed framework defines that attitude in terms of a specified amount of caution, thereby enabling data analysis at the level of caution desired and on the basis of prior information. The caution parameter represents the attitude toward partial prior information in much the same way as a loss function represents the attitude toward risk. When there is very little prior information and nonzero caution, the resulting inferences correspond to those of the candidate confidence intervals and p-values that are most similar to the credible intervals and hypothesis probabilities of the specified Bayesian posterior. On the other hand, in the presence of a known physical distribution of the parameter, inferences are based only on the corresponding physical posterior. In those extremes of either negligible prior information or complete prior information, inferences do not depend on the degree of caution. Partial prior information between those two extremes leads to intermediate inferences that are more frequentist to the extent that the caution is high and more Bayesian to the extent that the caution is low.

We study efficient nonparametric estimation of distribution functions of several scientifically meaningful sub-populations from data consisting of mixed samples where the sub-population identifiers are missing. Only probabilities of each observation belonging to a sub-population are available. The problem arises from several biomedical studies such as quantitative trait locus (QTL) analysis and genetic studies with ungenotyped relatives where the scientific interest lies in estimating the cumulative distribution function of a trait given a specific genotype. However, in these studies subjects’ genotypes may not be directly observed. The distribution of the trait outcome is therefore a mixture of several genotype-specific distributions. We characterize the complete class of consistent estimators which includes members such as one type of nonparametric maximum likelihood estimator (NPMLE) and least squares or weighted least squares estimators. We identify the efficient estimator in the class that reaches the semiparametric efficiency bound, and we implement it using a simple procedure that remains consistent even if several components of the estimator are mis-specified. In addition, our close inspections on two commonly used NPMLEs in these problems show the surprising results that the NPMLE in one form is highly inefficient, while in the other form is inconsistent. We provide simulation procedures to illustrate the theoretical results and demonstrate the proposed methods through two real data examples.

We study simultaneous rank procedures for unbalanced designs with independent observations. The hypotheses are formulated in terms of purely nonparametric treatment effects. In this context, we derive rank-based multiple contrast test procedures and simultaneous confidence intervals which take the correlation between the test statistics into account. Hereby, the individual test decisions and the simultaneous confidence intervals are compatible. This means, whenever an individual hypothesis has been rejected by the multiple contrast test, the corresponding simultaneous confidence interval does not include the null, i.e. the hypothetical value of no treatment effect. The procedures allow for testing arbitrary purely nonparametric multiple linear hypotheses (e.g. many-to-one, all-pairs, changepoint, or even average comparisons). We do not assume homogeneous variances of the data; in particular, the distributions can have different shapes even under the null hypothesis. Thus, a solution to the multiple nonparametric Behrens-Fisher problem is presented in this unified framework.

In this paper, we construct a family of nonparametric multivariate multisample tests based on depth rankings. These tests are of Kruskal-Wallis type in the sense that the samples are variously ordered. However, unlike the Kruskal-Wallis test, these tests are based upon a depth ranking using a statistical depth function such as the halfspace depth or the Mahalanobis depth, etc. The types of tests we propose are adapted to the depth function that is most appropriate for the application. Under the null hypothesis that all samples come from the same distribution, we show that the test statistic asymptotically has a chi-square distribution. Some comparisons of power are made with the Hotelling T², and the test of Choi and Marden (1997). Our test is particularly recommended when the data are of unknown distribution type where there is some evidence that the density contours are not elliptical. However, when the data are normally distributed, we often obtain high relative power.

We consider Bayesian estimation of the location parameter θ of a random vector X having a unimodal spherically symmetric density f(‖x−θ‖2) when the prior density π(‖θ‖2) is spherically symmetric and superharmonic. We study minimaxity of the generalized Bayes estimator δπ(X)=X+∇M(X)/m(X) under quadratic loss, where m is the marginal associated to f(‖x−θ‖2) and M is the marginal with respect to F(‖x−θ‖2)=1/2∫‖x−θ‖2∞f(t) dt under the condition inf t≥0F(t)/f(t)=c>0 (see Berger [1]). We adopt a common approach to the cases where F(t)/f(t) is nonincreasing or nondecreasing and, although details differ in the two settings, this paper complements the article by Fourdrinier and Strawderman [7] who dealt with only the case where F(t)/f(t) is nondecreasing. When F(t)/f(t) is nonincreasing, we show that the Bayes estimator is minimax provided a ‖∇π(‖θ‖2)‖2/π(‖θ‖2)+2 c2 Δπ(‖θ‖2)≤0 where a is a constant depending on the sampling density. When F(t)/f(t) is nondecreasing, the first term of that inequality is replaced by b g(‖θ‖2) where b also depends on f and where g(‖θ‖2) is a superharmonic upper bound of ‖∇π(‖θ‖2)‖2/π(‖θ‖2). Examples illustrate the theory.

This paper considers the problem of model selection in a nonparametric additive mixed modeling framework. The fixed effects are modeled nonparametrically using truncated series expansions with B-spline basis. Estimation and selection of such nonparametric fixed effects are simultaneously achieved by using the adaptive group lasso methodology, while the random effects are selected by a traditional backward selection mechanism. To facilitate the automatic selection of model dimension, computable expressions for the degrees of freedom for both the fixed and random effects components are derived, and the Bayesian Information criterion (BIC) is used to select the final model choice. Theoretically it is shown that this BIC model selection method is consistent, while computationally a practical algorithm is developed for solving the optimization problem involved. Simulation results show that the proposed methodology is often capable of selecting the correct significant fixed and random effects components, especially when the sample size and/or signal to noise ratio are not too small. The new method is also applied to two real data sets.

The present article presents a methodological advance which contributes to the area of nonparametric survival analysis under random right censoring. The central idea is to develop pointwise confidence intervals for the survival function by means of a central limit theorem for an, already existing in the literature, kernel smooth survival estimate. Numerical simulations reveal the progress in coverage accuracy offered by the suggested confidence intervals over the proposals already existing in the literature.

The paper concerns the problem of pointwise adaptive estimation in regression when the noise is heteroscedastic and incorrectly known. The use of the local approximation method, which includes the local polynomial smoothing as a particular case, leads to a finite family of estimators corresponding to different degrees of smoothing. Data-driven choice of localization degree in this case can be understood as the problem of selection from this family. This task can be performed by a suggested in Katkovnik and Spokoiny (2008) FLL technique based on Lepski’s method. An important issue with this type of procedures – the choice of certain tuning parameters – was addressed in Spokoiny and Vial (2009). The authors called their approach to the parameter calibration “propagation”. In the present paper the propagation approach is developed and justified for the heteroscedastic case in presence of the noise misspecification. Our analysis shows that the adaptive procedure allows a misspecification of the covariance matrix with a relative error of order (logn)⁻¹, where n is the sample size.

Studying the structure of so-called real networks, that is networks obtained from sociological or biological data for instance, has become a major field of interest in the last decade. One way to deal with it is to consider that networks are partially built from small functional units called motifs, which can be found by looking for small subgraphs whose numbers of occurrences in the whole network are surprisingly high. In this article, we propose to define motifs through a local over-representation in the network and develop a statistic to detect them without relying on simulations. We then illustrate the performance of our procedure on simulated and real data, recovering already known biologically relevant motifs. Moreover, we explain how our method gives some information about the respective roles of the vertices in a motif.

One-step ahead prediction for the multinomial model is considered. The performance of a predictive density is evaluated by the average Kullback-Leibler divergence from the true density to the predictive density. Asymptotic approximations of risk functions of Bayesian predictive densities based on Dirichlet priors are obtained. It is shown that a Bayesian predictive density based on a specific Dirichlet prior is asymptotically minimax. The asymptotically minimax prior is different from known objective priors such as the Jeffreys prior or the uniform prior.

Surrogate losses underlie numerous state-of-the-art binary classification algorithms, such as support vector machines and boosting. The impact of a surrogate loss on the statistical performance of an algorithm is well-understood in symmetric classification settings, where the misclassification costs are equal and the loss is a margin loss. In particular, classification-calibrated losses are known to imply desirable properties such as consistency. While numerous efforts have been made to extend surrogate loss-based algorithms to asymmetric settings, to deal with unequal misclassification costs or training data imbalance, considerably less attention has been paid to whether the modified loss is still calibrated in some sense. This article extends the theory of classification-calibrated losses to asymmetric problems. As in the symmetric case, it is shown that calibrated asymmetric surrogate losses give rise to excess risk bounds, which control the expected misclassification cost in terms of the excess surrogate risk. This theory is illustrated on the class of uneven margin losses, and the uneven hinge, squared error, exponential, and sigmoid losses are treated in detail.

Most statistical software packages implement numerical strategies for computation of maximum likelihood estimates in random effects models. Little is known, however, about the algebraic complexity of this problem. For the one-way layout with random effects and unbalanced group sizes, we give formulas for the algebraic degree of the likelihood equations as well as the equations for restricted maximum likelihood estimation. In particular, the latter approach is shown to be algebraically less complex. The formulas are obtained by studying a univariate rational equation whose solutions correspond to the solutions of the likelihood equations. Applying techniques from computational algebra, we also show that balanced two-way layouts with or without interaction have likelihood equations of degree four. Our work suggests that algebraic methods allow one to reliably find global optima of likelihood functions of linear mixed models with a small number of variance components.

In a 2003 paper, Hjort and Claeskens proposed a framework for studying the limiting distributions and asymptotic risk properties of model average estimators under parametric models. They also suggested a simple method for constructing confidence intervals for the parameters of interest estimated by model averaging. The purpose of this paper is to broaden the scope of the aforementioned study to include a semi-parametric varying-coefficient partially linear measurement error model. Within this context, we develop a model averaging scheme for the unknowns, derive the model average estimator’s asymptotic distribution, and develop a confidence interval procedure of the unknowns with an actual coverage probability that tends toward the nominal level in large samples. We further show that confidence intervals that are constructed based on the model average estimators are asymptotically the same as those obtained under the full model. A simulation study examines the finite sample performance of the model average estimators, and a real data analysis illustrates the application of the method in practice.

In this paper, the empirical likelihood inferences for partially time-varying coefficient errors-in-variables model with dependent observations are investigated. We propose an empirical log-likelihood ratio function for the regression parameters and show that its limiting distribution is a mixture of central chi-squared distributions. In order that the Wilks’ phenomenon holds, we construct an adjusted empirical log-likelihood ratio for the regression parameters. The adjusted empirical log-likelihood is shown to have a standard chi-squared limiting distribution. Simulations show that the proposed confidence regions have satisfactory performance.

We define a new measure of variable importance of an exposure on a continuous outcome, accounting for potential confounders. The exposure features a reference level $x_{0}$ with positive mass and a continuum of other levels. For the purpose of estimating it, we fully develop the semi-parametric estimation methodology called targeted minimum loss estimation methodology (TMLE) [23,22]. We cover the whole spectrum of its theoretical study (convergence of the iterative procedure which is at the core of the TMLE methodology; consistency and asymptotic normality of the estimator), practical implementation, simulation study and application to a genomic example that originally motivated this article. In the latter, the exposure $X$ and response $Y$ are, respectively, the DNA copy number and expression level of a given gene in a cancer cell. Here, the reference level is $x_{0}=2$, that is the expected DNA copy number in a normal cell. The confounder is a measure of the methylation of the gene. The fact that there is no clear biological indication that $X$ and $Y$ can be interpreted as an exposure and a response, respectively, is not problematic.

Exponential-family random graph models (ERGMs) provide a principled and flexible way to model and simulate features common in social networks, such as propensities for homophily, mutuality, and friend-of-a-friend triad closure, through choice of model terms (sufficient statistics). However, those ERGMs modeling the more complex features have, to date, been limited to binary data: presence or absence of ties. Thus, analysis of valued networks, such as those where counts, measurements, or ranks are observed, has necessitated dichotomizing them, losing information and introducing biases.

In this work, we generalize ERGMs to valued networks. Focusing on modeling counts, we formulate an ERGM for networks whose ties are counts and discuss issues that arise when moving beyond the binary case. We introduce model terms that generalize and model common social network features for such data and apply these methods to a network dataset whose values are counts of interactions.

This paper is concerned with the problem of goodness-of-fit for curve registration, and more precisely for the shifted curve model, whose application field reaches from computer vision and road traffic prediction to medicine. We give bounds for the asymptotic minimax separation rate, when the functions in the alternative lie in Sobolev balls and the separation from the null hypothesis is measured by the $l_{2}$-norm. We use the generalized likelihood ratio to build a nonadaptive procedure depending on a tuning parameter, which we choose in an optimal way according to the smoothness of the ambient space. Then, a Bonferroni procedure is applied to give an adaptive test over a range of Sobolev balls. Both achieve the asymptotic minimax separation rates, up to possible logarithmic factors.

The intensity of a Gibbs point process model is usually an intractable function of the model parameters. This is a severe restriction on the practical application of such models. We develop a new approximation for the intensity of a stationary Gibbs point process on $\mathbb{R}^{d}$. For pairwise interaction processes, the approximation can be computed rapidly and is surprisingly accurate. The new approximation is qualitatively similar to the mean field approximation, but is far more accurate, and does not exhibit the same pathologies. It may be regarded as a counterpart of the Percus-Yevick approximation.

This paper is concerned with the problem of flexibly estimating the conditional density of a response variable given covariates. In our approach the density is modeled as a mixture of heteroscedastic normals with the means, variances and mixing probabilities all varying smoothly as functions of the covariates. We use the variational Bayes approach and propose a novel fast algorithm for simultaneous covariate selection, component selection and parameter estimation. Our method is able to deal with the local maxima problem inherent in mixture model fitting, and is applicable to high-dimensional settings where the number of covariates can be larger than the sample size. In the special case of the classical regression model, the proposed algorithm is similar to currently used greedy algorithms while having many attractive properties and working efficiently in high-dimensional problems. The methodology is demonstrated through simulated and real examples.

We discuss efficient estimation in regression models that are defined by a finite-dimensional parametric constraint. This includes a variety of regression models, in particular the basic nonlinear regression model and quasi-likelihood regression. We are interested in the case where responses are missing at random. This is a popular research topic and various methods have been proposed in the literature. However, many of them are complicated and are not shown to be efficient. The method presented here is, in contrast, very simple – we use an estimating equation that does not impute missing responses – and we also prove that it is efficient if an appropriate weight matrix is selected. Finally, we show that this weight matrix can be replaced by a consistent estimator without losing the efficiency property.

Varying coefficient (VC) models are commonly used to study dynamic patterns in many scientific areas. In particular, VC models in quantile regression are known to provide a more complete description of the response distribution than in mean regression. In this paper, we develop a variable selection method for VC models in quantile regression using a shrinkage idea. The proposed method is based on the basis expansion of each varying coefficient and the regularization penalty on the Euclidean norm of the corresponding coefficient vector. We show that our estimator is obtained as an optimal solution to the second order cone programming (SOCP) problem and that the proposed procedure has consistency in variable selection under suitable conditions. Further, we show that the estimated relevant coefficients converge to the true functions at the univariate optimal rate. Finally, the method is illustrated with numerical simulations including the analysis of forced expiratory volume (FEV) data.

We advocate the decomposition of goodness of fit into contributions of (groups of) regressor variables according to the Shapley value or—if regressors are exogenously grouped—the Owen value because of the attractive axioms associated with these values. A wage regression model with German data illustrates the method.

The Luria-Delbrück distribution is a classical model of mutations in cell kinetics. It is obtained as a limit when the probability of mutation tends to zero and the number of divisions to infinity. It can be interpreted as a compound Poisson distribution (for the number of mutations) of exponential mixtures (for the developing time of mutant clones) of geometric distributions (for the number of cells produced by a mutant clone in a given time). The probabilistic interpretation, and a rigourous proof of convergence in the general case, are deduced from classical results on Bellman-Harris branching processes. The two parameters of the Luria-Delbrück distribution are the expected number of mutations, which is the parameter of interest, and the relative fitness of normal cells compared to mutants, which is the heavy tail exponent. Both can be simultaneously estimated by the maximum likehood method. However, the computation becomes numerically unstable when the maximal value of the sample is large, which occurs frequently due to the heavy tail property. Based on the empirical probability generating function, robust estimators are proposed and their asymptotic variance is given. They are comparable in precision to maximum likelihood estimators, with a much broader range of calculability, a better numerical stability, and a negligible computing time.

In this paper the interest is to estimate the dependence between two variables conditionally upon a covariate, through copula modelling. In recent literature nonparametric estimators for conditional copula functions in case of a univariate covariate have been proposed. The aim of this paper is to nonparametrically estimate a conditional copula when the covariate takes on values in more complex spaces. We consider multivariate covariates and functional covariates. We establish weak convergence, and bias and variance properties of the proposed nonparametric estimators. We also briefly discuss nonparametric estimation of conditional association measures such as a conditional Kendall’s tau. The case of functional covariates is of particular interest and challenge, both from theoretical as well as practical point of view. For this setting we provide an illustration with a real data example in which the covariates are spectral curves. A simulation study investigating the finite-sample performances of the discussed estimators is provided.

We prove the weak consistency of the posterior distribution and that of the Bayes estimator for a two-phase piecewise linear regression model where the break-point is unknown. We also establish a Bernstein-von Mises theorem for this non regular model. The non differentiability of the likelihood of the model with regard to the break-point parameter induces technical difficulties that we overcome by creating a regularised version of the problem at hand. We first recover the strong consistency of the quantities of interest for the regularised version, using results about the MLE, and we then prove that the regularised version and the original version of the problem share the same asymptotic properties.

We consider nonparametric ranking methods for matched pairs, whose distributions can have different shapes even under the null hypothesis of no treatment effect. Although the data may not be exchangeable under the null, we investigate a permutation approach as a valid procedure for finite sample sizes. In particular, we derive the limit of the studentized permutation distribution under alternatives, which can be used for the construction of $(1-\alpha)$-confidence intervals. Simulation studies show that the new approach is more accurate than its competitors. The procedures are illustrated using a real data set.

We consider nonparametric functional regression when both predictors and responses are functions. More specifically, we let $(X_{1},Y_{1}),\ldots, (X_{n},Y_{n})$ be random elements in $\mathcal{F}\times\mathcal{H}$ where $\mathcal{F}$ is a semi-metric space and $\mathcal{H}$ is a separable Hilbert space. Based on a recently introduced notion of weak dependence for functional data, we showed the almost sure convergence rates of both the Nadaraya-Watson estimator and the nearest neighbor estimator, in a unified manner. Several factors, including functional nature of the responses, the assumptions on the functional variables using the Orlicz norm and the desired generality on weakly dependent data, make the theoretical investigations more challenging and interesting.

We estimate the self-similarity index of a $H$-sssi process through complex variations. The advantage of the complex variations is that they do not require existence of moments and can therefore be used for infinite variance processes.

We study the problem of detection of high-dimensional signal functions in the Gaussian white noise model. We assume that, in addition to a smoothness assumption, the signal function has an additive sparse structure. The detection problem is expressed in terms of a nonparametric hypothesis testing problem and is solved using asymptotically minimax approach. We provide minimax test procedures that are adaptive in the sparsity parameter in the high sparsity case. We extend some known results related to the detection of sparse high-dimensional vectors to the functional case. In particular, our derivation of asymptotic detection rates is based on same detection boundaries as in the vector case.

This paper introduces nonparametric estimation methods for spatial point processes satisfying a particular renewal property. Martingale methods yield a unified approach for renewal processes in both one and two dimensions, and can be used for both synchronous and asynchronous data. In each case, we obtain martingale estimators of the avoidance probabilities that characterize the renewal process. Asymptotic properties of the estimators are studied analytically and empirically.

More than 50 methods have been developed to draw unequal probability samples with fixed sample size. All these methods require the sum of the inclusion probabilities to be an integer number. There are cases, however, where the sum of desired inclusion probabilities is not an integer. Then, classical algorithms for drawing samples cannot be directly applied. We present two methods to overcome the problem of sample selection with unequal inclusion probabilities when their sum is not an integer and the sample size cannot be fixed. The first one consists in splitting the inclusion probability vector. The second method is based on extending the population with a phantom unit. For both methods the sample size is almost fixed, and equal to the integer part of the sum of the inclusion probabilities or this integer plus one.

Confidence bands are confidence sets for an unknown function $f$, containing all functions within some sup-norm distance of an estimator. In the density estimation, regression, and white noise models, we consider the problem of constructing adaptive confidence bands, whose width contracts at an optimal rate over a range of Hölder classes.

While adaptive estimators exist, in general adaptive confidence bands do not, and to proceed we must place further conditions on $f$. We discuss previous approaches to this issue, and show it is necessary to restrict $f$ to fundamentally smaller classes of functions.

We then consider the self-similar functions, whose Hölder norm is similar at large and small scales. We show that such functions may be considered typical functions of a given Hölder class, and that the assumption of self-similarity is both necessary and sufficient for the construction of adaptive bands.

This paper identifies and develops the class of Gaussian copula models for marginal regression analysis of non-normal dependent observations. The class provides a natural extension of traditional linear regression models with normal correlated errors. Any kind of continuous, discrete and categorical responses is allowed. Dependence is conveniently modelled in terms of multivariate normal errors. Inference is performed through a likelihood approach. While the likelihood function is available in closed-form for continuous responses, in the non-continuous setting numerical approximations are used. Residual analysis and a specification test are suggested for validating the adequacy of the assumed multivariate model. Methodology is implemented in a R package called gcmr. Illustrations include simulations and real data applications regarding time series, cross-design data, longitudinal studies, survival analysis and spatial regression.

Given two probability measures, $\mathbb{P}$ and $\mathbb{Q}$ defined on a measurable space, $S$, the integral probability metric (IPM) is defined as $$\gamma_{\mathcal{F}}(\mathbb{P},\mathbb{Q})=\sup\left\{\left\vert \int_{S}f\,d\mathbb{P}-\int_{S}f\,d\mathbb{Q}\right\vert\,:\,f\in\mathcal{F}\right\},$$ where $\mathcal{F}$ is a class of real-valued bounded measurable functions on $S$. By appropriately choosing $\mathcal{F}$, various popular distances between $\mathbb{P}$ and $\mathbb{Q}$, including the Kantorovich metric, Fortet-Mourier metric, dual-bounded Lipschitz distance (also called the Dudley metric), total variation distance, and kernel distance, can be obtained.

In this paper, we consider the problem of estimating $\gamma_{\mathcal{F}}$ from finite random samples drawn i.i.d. from $\mathbb{P}$ and $\mathbb{Q}$. Although the above mentioned distances cannot be computed in closed form for every $\mathbb{P}$ and $\mathbb{Q}$, we show their empirical estimators to be easily computable, and strongly consistent (except for the total-variation distance). We further analyze their rates of convergence. Based on these results, we discuss the advantages of certain choices of $\mathcal{F}$ (and therefore the corresponding IPMs) over others—in particular, the kernel distance is shown to have three favorable properties compared with the other mentioned distances: it is computationally cheaper, the empirical estimate converges at a faster rate to the population value, and the rate of convergence is independent of the dimension $d$ of the space (for $S=\mathbb{R}^{d}$). We also provide a novel interpretation of IPMs and their empirical estimators by relating them to the problem of binary classification: while the IPM between class-conditional distributions is the negative of the optimal risk associated with a binary classifier, the smoothness of an appropriate binary classifier (e.g., support vector machine, Lipschitz classifier, etc.) is inversely related to the empirical estimator of the IPM between these class-conditional distributions.

Statistical models that possess symmetry arise in diverse settings such as random fields associated to geophysical phenomena, exchangeable processes in Bayesian statistics, and cyclostationary processes in engineering. We formalize the notion of a symmetric model via group invariance. We propose projection onto a group fixed point subspace as a fundamental way of regularizing covariance matrices in the high-dimensional regime. In terms of parameters associated to the group we derive precise rates of convergence of the regularized covariance matrix and demonstrate that significant statistical gains may be expected in terms of the sample complexity. We further explore the consequences of symmetry in related model-selection problems such as the learning of sparse covariance and inverse covariance matrices. We also verify our results with simulations.

In this paper I review covariance-based Partial Least Squares (PLS) methods, focusing on common features of their respective algorithms and optimization criteria. I then show how these algorithms can be adjusted for use as optimal scaling tools. Three new PLS-type algorithms are proposed for the analysis of one, two or several blocks of variables: the Non-Metric NIPALS, the Non-Metric PLS Regression and the Non-Metric PLS Path Modeling, respectively. These algorithms extend the applicability of PLS methods to data measured on different measurement scales, as well as to variables linked by non-linear relationships.

In this paper we study a nonparametric estimation problem for stochastic feedforward systems with nodes of type G/G/$\infty.$ We assume that we have observations of the external arrival and external departure processes of customers of the system, but no information about the movements of the indistinguishable customers in the network. Our aim is the construction of estimators for the characteristic functions and the densities of the service time distributions at the nodes as well as for the routing probabilities. Since the system is only partly observed we have to clarify first if the parameters are identifiable from the given data. The crucial point in our approach is the observation that in our stochastic networks under study the influence of the arrival processes on the departure processes can be described in a linear and time-invariant model. This makes it possible to apply cross-spectral techniques for multivariate point processes. The construction of the estimators is then based on smoothed periodograms. We prove asymptotic normality for the estimators. We present the statistical analysis for a tandem system of two nodes in full details and show afterwards how the results can be generalized to feedforward systems of three or more nodes and to systems with positive feedback probabilities at the nodes.

We address the estimation of conditional quantiles when the covariate is functional and when the order of the quantiles converges to one as the sample size increases. In a first time, we investigate to what extent these large conditional quantiles can still be estimated through a functional kernel estimator of the conditional survival function. Sufficient conditions on the rate of convergence of their order to one are provided to obtain asymptotically Gaussian distributed estimators. In a second time, basing on these result, a functional Weissman estimator is derived, permitting to estimate large conditional quantiles of arbitrary large order. These results are illustrated on finite sample situations.

The minimax rate for estimating the regression function $r(\cdot)=\mathbb{E}(y|X=\cdot)$ when $y\in\mathbb{R}$ and $X$ takes values in a function space is the initial motivation of this work. Recent articles underline the major role of the shifted small ball probability $\mathbb{P}(\Vert X-x_{0}\Vert <\cdot)$ in the variance of classical estimates. The main results are twofold. First, starting from a theorem by [41], we study the small ball probability $\mathbb{P}(S<\varepsilon)$ when $\varepsilon\downarrow0$ with $S=\sum_{i=1}^{+\infty}\lambda_{i}Z_{i}$ where the $Z_{i}$’s are i.i.d. positive and $(\lambda_{i})_{i\in\mathbb{N}}$ a positive nonincreasing sequence such that $\sum\lambda_{i}<+\infty$. It is shown that $\mathbb{P}(S<\cdot)$ belongs to a class of functions introduced by de Haan, well-known in extreme value theory, the class of Gamma-varying functions, for which an exponential-integral representation is available. Second this approach allows to derive minimax lower bounds for the risk at a fixed point $x_{0}$ when $X\in\mathcal{H}$ some Hilbert space of functions. Denoting this minimax risk: \[\mathcal{R}_{n}^{\ast}=\inf_{T_{n}}\sup_{r\in\mathcal{E}}\mathbb{E}\left\vert T_{n}-r(x_{0})\right\vert ^{2}\] where $T_{n}$ is any estimate of $r(x_{0})$ and $\mathcal{E}$ is some class of smooth functions from $\mathcal{H}$ to $\mathbb{R}$ it turns out that, in a general framework, $n^{\tau}\mathcal{R}_{n}^{\ast}\rightarrow+\infty$ for any $\tau>0.$ This negative result may pave the way towards new approaches for modeling regression with functional data.

We investigate two models for the following setup: We consider a stochastic process $\boldsymbol{X}\in C[0,1]$ whose distribution belongs to a parametric family indexed by $\vartheta\in\Theta\subset \mathbb{R} $. In case $\vartheta=0$, $\boldsymbol{X}$ is a generalized Pareto process. Based on $n$ independent copies $\boldsymbol{X}^{(1)},\dots,\boldsymbol{X}^{(n)}$ of $\boldsymbol{X}$, we establish local asymptotic normality (LAN) of the point process of exceedances among $\boldsymbol{X}^{(1)},\dots,\boldsymbol{X}^{(n)}$ above an increasing threshold line in each model.

The corresponding central sequences provide asymptotically optimal sequences of tests for testing $H_{0}:\vartheta=0$ against a sequence of alternatives $H_{n}:\vartheta=\vartheta_{n}$ converging to zero as $n$ increases. In one model, with an underlying exponential family, the central sequence is provided by the number of exceedances only, whereas in the other one the exceedances themselves contribute, too. However it turns out that, in both cases, the test statistics also depend on some additional and usually unknown model parameters.

We, therefore, consider an omnibus test statistic sequence as well and compute its asymptotic relative efficiency with respect to the optimal test sequence.

We prove oracle inequalities for three different types of adaptation procedures inspired by cross-validation and aggregation. These procedures are then applied to the construction of Lasso estimators and aggregation with exponential weights with data-driven regularization and temperature parameters, respectively. We also prove oracle inequalities for the cross-validation procedure itself under some convexity assumptions.

Binary data with high-dimensional covariates have become more and more common in many disciplines. In this paper we consider the maximum likelihood estimation for logistic regression models with a diverging number of covariates. Under mild conditions we establish the asymptotic normality of the maximum likelihood estimate when the number of covariates $p$ goes to infinity with the sample size $n$ in the order of $p=o(n)$. This remarkably improves the existing results that can only allow $p$ growing in an order of $o(n^{\alpha})$ with $\alpha\in[1/5,1/2]$ [12, 14]. A major innovation in our proof is the use of the injective function.

The stochastic block model (SBM) is a probabilistic model designed to describe heterogeneous directed and undirected graphs. In this paper, we address the asymptotic inference in SBM by use of maximum-likelihood and variational approaches. The identifiability of SBM is proved while asymptotic properties of maximum-likelihood and variational estimators are derived. In particular, the consistency of these estimators is settled for the probability of an edge between two vertices (and for the group proportions at the price of an additional assumption), which is to the best of our knowledge the first result of this type for variational estimators in random graphs.

This paper gives a general method for constructing quasi-Latin square, quasi-Latin rectangle and extended quasi-Latin rectangle designs for symmetric factorial experiments. Two further methods are given for parameter values satisfying certain conditions. The construction of designs for a range of numbers of rows and columns is discussed so that the different construction techniques are covered. For some row and column combinations, different designs are compared. The construction of designs with rows and columns that are nested or contiguous is also discussed.

The statistical analysis of data belonging to Riemannian manifolds is becoming increasingly important in many applications. The aim of this work is to introduce models for spatial dependence among Riemannian data, with a special focus on the case of positive definite symmetric matrices. First, the Riemannian semivariogram of a field of positive definite symmetric matrices is defined. Then, we propose an estimator for the mean which considers both the non Euclidean nature of the data and their spatial correlation. Simulated data are used to evaluate the performance of the proposed estimator: taking into account spatial dependence leads to better estimates when observations are irregularly spaced in the region of interest. Finally, we address a meteorological problem, namely, the estimation of the covariance matrix between temperature and precipitation for the province of Quebec in Canada.

It is often the case that there are several studies measuring the same parameter. Naturally, it is of interest to provide a systematic way to combine the information from these studies. Examples of such situations include clinical trials, key comparison trials and other problems of practical importance. Singh et al. (2005) provide a compelling framework for combining information from multiple sources using the framework of confidence distributions. In this paper we investigate the feasibility of using the Dempster-Shafer recombination rule on this problem. We derive a practical combination rule and show that under assumption of asymptotic normality, the Dempster-Shafer combined confidence distribution is asymptotically equivalent to one of the method proposed in Singh et al. (2005). Numerical studies and comparisons for the common mean problem and the odds ratio in $2\times 2$ tables are included.

This paper is devoted to rejective sampling. We provide an expansion of joint inclusion probabilities of any order in terms of the inclusion probabilities of order one, extending previous results by Hájek (1964) and Hájek (1981) and making the remainder term more precise. Following Hájek (1981), the proof is based on Edgeworth expansions. The main result is applied to derive bounds on higher order correlations, which are needed for the consistency and asymptotic normality of several complex estimators.

We investigate posterior contraction rates for priors on multivariate functions that are constructed using tensor-product B-spline expansions. We prove that using a hierarchical prior with an appropriate prior distribution on the partition size and Gaussian prior weights on the B-spline coefficients, procedures can be obtained that adapt to the degree of smoothness of the unknown function up to the order of the splines that are used. We take a unified approach including important nonparametric statistical settings like density estimation, regression, and classification.

Motivated by fluorescence lifetime measurements, this paper considers the problem of nonparametric density estimation in the pile-up model, where observations suffer also from measurement errors. In the pile-up model, an observation is defined as the minimum of a random number of i.i.d. variables following the target distribution. Adaptive nonparametric estimators are proposed for this pile-up model with measurement errors. Furthermore, oracle type risk bounds for the mean integrated squared error (MISE) are provided. Finally, the estimation method is assessed by a simulation study and the application to real fluorescence lifetime data.

We introduce a novel, Bayesian nonparametric, infinite-mixture regression model. The model has unimodal kernel (component) densities, and has covariate-dependent mixture weights that are defined by an infinite ordered-category probits regression. Based on these mixture weights, the regression model predicts a probability density that becomes increasingly unimodal as the explanatory power of the covariate (vector) increases, and increasingly multimodal as this explanatory power decreases, while allowing the explanatory power to vary from one covariate (vector) value to another. The model is illustrated and compared against many other regression models in terms of predictive performance, through the analysis of many real and simulated data sets.

We study the problem of estimating a temporally varying coefficient and varying structure (VCVS) graphical model underlying data collected over a period of time, such as social states of interacting individuals or microarray expression profiles of gene networks, as opposed to i.i.d. data from an invariant model widely considered in current literature of structural estimation. In particular, we consider the scenario in which the model evolves in a piece-wise constant fashion. We propose a procedure that estimates the structure of a graphical model by minimizing the temporally smoothed L1 penalized regression, which allows jointly estimating the partition boundaries of the VCVS model and the coefficient of the sparse precision matrix on each block of the partition. A highly scalable proximal gradient method is proposed to solve the resultant convex optimization problem; and the conditions for sparsistent estimation and the convergence rate of both the partition boundaries and the network structure are established for the first time for such estimators.

Partially identified models typically involve set identification rather than point identification. That is, the distribution of observables is consistent with a set of values for the target parameter, rather than a single value. Interval estimation procedures therefore behave differently than for identified models. For instance, a Bayesian credible set arising from a proper prior distribution will tend to a non-degenerate set as the sample size goes to infinity. A natural question arising is for what parameter values does the limit of the Bayesian credible interval fail to cover its target? Intuition suggests this would arise for parameter values which are not very consistent with the prior distribution. The aim of this paper is to quantify this intuition.

The graphical lasso [5] is an algorithm for learning the structure in an undirected Gaussian graphical model, using $\ell_{1}$ regularization to control the number of zeros in the precision matrix $\boldsymbol{\Theta}=\boldsymbol{\Sigma}^{-1}$ [2, 11]. The R package GLASSO [5] is popular, fast, and allows one to efficiently build a path of models for different values of the tuning parameter. Convergence of GLASSO can be tricky; the converged precision matrix might not be the inverse of the estimated covariance, and occasionally it fails to converge with warm starts. In this paper we explain this behavior, and propose new algorithms that appear to outperform GLASSO.

By studying the “normal equations” we see that, GLASSO is solving the dual of the graphical lasso penalized likelihood, by block coordinate ascent; a result which can also be found in [2]. In this dual, the target of estimation is $\boldsymbol{\Sigma}$, the covariance matrix, rather than the precision matrix $\boldsymbol{\Theta}$. We propose similar primal algorithms P-GLASSO and DP-GLASSO, that also operate by block-coordinate descent, where $\boldsymbol{\Theta}$ is the optimization target. We study all of these algorithms, and in particular different approaches to solving their coordinate sub-problems. We conclude that DP-GLASSO is superior from several points of view.

For the zone of moderate deviation probabilities the local asymptotic minimax lower bound of asymptotic efficiency of estimators is established. The estimation parameter is multidimensional. The lower bound admits the interpretation as the lower bound of asymptotic efficiency in confidence estimation.

We consider quasi maximum likelihood (QML) estimation for general non-Gaussian discrete-time linear state space models and equidistantly observed multivariate Lévy-driven continuous-time autoregressive moving average (MCARMA) processes. In the discrete-time setting, we prove strong consistency and asymptotic normality of the QML estimator under standard moment assumptions and a strong-mixing condition on the output process of the state space model. In the second part of the paper, we investigate probabilistic and analytical properties of equidistantly sampled continuous-time state space models and apply our results from the discrete-time setting to derive the asymptotic properties of the QML estimator of discretely recorded MCARMA processes. Under natural identifiability conditions, the estimators are again consistent and asymptotically normally distributed for any sampling frequency. We also demonstrate the practical applicability of our method through a simulation study and a data example from econometrics.

In this paper, we formulate the partially linear single-index models as bi-index dimension reduction models for the purpose of identifying significant covariates in both the linear part and the single-index part through only one combined index in a dimension reduction approach. This is different from all existing dimension reduction methods in the literature, which in general identify two basis directions in the subspace spanned by the parameter vectors of interest, rather than the two parameter vectors themselves. This approach makes the identification and the subsequent estimation and variable selection easier than existing methods for multi-index models. When the number of parameters diverges with the sample size, we then adopt coordinate-independent sparse estimation procedure to select significant covariates and estimate the corresponding parameters. The resulting sparse dimension reduction estimators are shown to be consistent and asymptotically normal with the oracle property. Simulations are conducted to evaluate the performance of the proposed method, and a real data set is analysed for an illustration.

We consider linear inverse problems in a nonparametric statistical framework. Both the signal and the operator are unknown and subject to error measurements. We establish minimax rates of convergence under squared error loss when the operator admits a blockwise singular value decomposition (blockwise SVD) and the smoothness of the signal is measured in a Sobolev sense. We construct a nonlinear procedure adapting simultaneously to the unknown smoothness of both the signal and the operator and achieving the optimal rate of convergence to within logarithmic terms. When the noise level in the operator is dominant, by taking full advantage of the blockwise SVD property, we demonstrate that the block SVD procedure outperforms classical methods based on Galerkin projection [14] or nonlinear wavelet thresholding [18]. We subsequently apply our abstract framework to the specific case of blind deconvolution on the torus and on the sphere.

The Gaussian Graphical Model (GGM) is a popular tool for incorporating sparsity into joint multivariate distributions. The G-Wishart distribution, a conjugate prior for precision matrices satisfying general GGM constraints, has now been in existence for over a decade. However, due to the lack of a direct sampler, its use has been limited in hierarchical Bayesian contexts, relegating mixing over the class of GGMs mostly to situations involving standard Gaussian likelihoods. Recent work has developed methods that couple model and parameter moves, first through reversible jump methods and later by direct evaluation of conditional Bayes factors and subsequent resampling. Further, methods for avoiding prior normalizing constant calculations–a serious bottleneck and source of numerical instability–have been proposed. We review and clarify these developments and then propose a new methodology for GGM comparison that blends many recent themes. Theoretical developments and computational timing experiments reveal an algorithm that has limited computational demands and dramatically improves on computing times of existing methods. We conclude by developing a parsimonious multivariate stochastic volatility model that embeds GGM uncertainty in a larger hierarchical framework. The method is shown to be capable of adapting to swings in market volatility, offering improved calibration of predictive distributions.

The definitions of $\Phi$ optimality and $\Phi$ admissibility of stochastic regression coefficients are given in a general multivariate random effects model under the generalized balanced loss function. $\Phi$ admissibility of linear estimators of stochastic regression coefficients is investigated. Sufficient and necessary conditions for linear estimators to be $\Phi$ admissible in classes of homogeneous and nonhomogeneous linear estimators are obtained, respectively.

Consider estimating an $n\times p$ matrix of means $\Theta$, say, from an $n\times p$ matrix of observations $X$, where the elements of $X$ are assumed to be independently normally distributed with $E(x_{ij})=\theta_{ij}$ and constant variance, and where the performance of an estimator is judged using a $p\times p$ matrix quadratic error loss function. A matrix version of the James-Stein estimator is proposed, depending on a tuning constant $a$. It is shown to dominate the usual maximum likelihood estimator for some choices of $a$ when $n\geq 3$. This result also extends to other shrinkage estimators and settings.

We describe a previously unnoted problem which, if it occurs, causes the empirical likelihood method to break down. It is related to the empty set problem, recently described in detail by Grendár and Judge (2009), which is the problem that the empirical likelihood model is empty, so that maximum empirical likelihood estimates do not exist. An example is the model that the mean is zero, while all observations are positive. A related problem, which appears to have gone unnoted so far, is what we call the zero likelihood problem. This occurs when the empirical likelihood model is nonempty but all its elements have zero empirical likelihood. Hence, also in this case inference regarding the model under investigation breaks down. An example is the model that the covariance is zero, and the sample consists of monotonically associated observations. In this paper, we define the problem generally and give examples. Although the problem can occur in many situations, we found it to be especially prevalent in marginal modeling of categorical data, when the problem often occurs with probability close to one for large, sparse contingency tables.

This paper provides a Bayesian procedure to aggregate experts’ information in a group decision making context. The belief of each expert is elicited as a multivariate prior distribution. Then, linear and logarithmic combination methods are used to represent a consensus distribution. Anyway, the choice of the appropriate strategy will depend on the decision maker’s judgements. A significant task when using opinion pooling is to find the optimal weights. In order to carry it out, a criterion based on Kullback-Leibler divergence is proposed. Furthermore, based on the previous idea, an alternative procedure is presented when a solution cannot be found. The theoretical foundations are discussed in detail for each aggregation scheme. In particular, it is shown that a general unified method is achieved when they are applied to multivariate natural exponential families. Finally, two illustrative examples show that the proposed techniques can be easily applied in practice and their usefulness for decision making under the described situations.

This paper is first devoted to the study of an adaptive wavelet-based estimator of the long-memory parameter for linear processes in a general semiparametric frame. As such this is an extension of the previous contribution of Bardet et al. (2008) which only concerned Gaussian processes. Moreover, the definition of the long-memory parameter estimator has been modified and the asymptotic results are improved even in the Gaussian case. Finally an adaptive goodness-of-fit test is also built and easy to be employed: it is a chi-square type test. Simulations confirm the interesting properties of consistency and robustness of the adaptive estimator and test.

In this paper, we consider a regression model built on dependent variables. This regression modelizes an input output relationship. Under boundedness type assumptions on the joint density function of the input variables, we show that a generalized Hoeffding-Sobol decomposition is available. This leads to new indices measuring the sensitivity of the output with respect to the input variables. We also study and discuss the estimation of these new indices.

For discrete data especially, standard P-values can misreport the true significance, even for moderately large sample sizes. The bootstrap P-value is the exact tail probability of an appropriate test statistic, calculated assuming the nuisance parameter equals the null maximum likelihood (ML) estimate. For basic discrete models and standard test statistics, bootstrap P-values have been found to be extremely close to uniformly distributed under the null ([1]). Detailed numerical results reported there suggest that this phenomenon is not explained by asymptotics. In this paper, we identify several desirable non-asymptotic properties of bootstrap P-values and provide arguments for why bootstrap P-values are so close to exact. The most important of these is that bootstrap will correct ‘incorrect’ ordering of the sample space and that this leads to a more pivotal distribution. Most of these arguments only hold for discrete models and when the null ML estimate is used.

Most common regression models for analyzing binary random variables are logistic and probit regression models. However it is well known that the estimates of regression coefficients for these models are not robust to outliers [26]. The robit regression model [1, 16] is a robust alternative to the probit and logistic models. The robit model is obtained by replacing the normal (logistic) distribution underlying the probit (logistic) regression model with the Student’s $t-$distribution. We consider a Bayesian analysis of binary data with the robit link function. We construct a data augmentation (DA) algorithm that can be used to explore the corresponding posterior distribution. Following [10] we further improve the DA algorithm by adding a simple extra step to each iteration. Though the two algorithms are basically equivalent in terms of computational complexity, the second algorithm is theoretically more efficient than the DA algorithm. Moreover, we analyze the convergence rates of these Markov chain Monte Carlo (MCMC) algorithms. We prove that, under certain conditions, both algorithms converge at a geometric rate. The geometric convergence rate has important theoretical and practical ramifications. Indeed, the geometric ergodicity guarantees that the ergodic averages used to approximate posterior expectations satisfy central limit theorems, which in turn allows for the construction of asymptotically valid standard errors. These standard errors can be used to choose an appropriate (Markov chain) Monte Carlo sample size and allow one to use the MCMC algorithms developed in this paper with the same level of confidence that one would have using classical (iid) Monte Carlo. The results are illustrated using a simple numerical example.

We estimate linear functionals in the classical deconvolution problem by kernel estimators. We obtain a uniform central limit theorem with $\sqrt{n}$–rate on the assumption that the smoothness of the functionals is larger than the ill–posedness of the problem, which is given by the polynomial decay rate of the characteristic function of the error. The limit distribution is a generalized Brownian bridge with a covariance structure that depends on the characteristic function of the error and on the functionals. The proposed estimators are optimal in the sense of semiparametric efficiency. The class of linear functionals is wide enough to incorporate the estimation of distribution functions. The proofs are based on smoothed empirical processes and mapping properties of the deconvolution operator.

Varying coefficient models are useful extensions of classical linear models. In practice, some of the coefficients may be just constant, while other coefficients are varying. Several methods have been developed to utilize the information that some coefficient functions are constant to improve estimation efficiency. However, in order for such methods to really work, the information about which coefficient functions are constant should be given in advance. In this paper, we propose a computationally efficient method to discriminate in a consistent way the constant coefficient functions from the varying ones. Additionally, we compare the performance of our proposal with that of previous methods developed for the same purpose in terms of model selection accuracy and computing time.

A recursive estimator of the conditional geometric median in Hilbert spaces is studied. It is based on a stochastic gradient algorithm whose aim is to minimize a weighted $L_{1}$ criterion and is consequently well adapted for robust online estimation. The weights are controlled by a kernel function and an associated bandwidth. Almost sure convergence and $L^{2}$ rates of convergence are proved under general conditions on the conditional distribution as well as the sequence of descent steps of the algorithm and the sequence of bandwidths. Asymptotic normality is also proved for the averaged version of the algorithm with an optimal rate of convergence. A simulation study confirms the interest of this new and fast algorithm when the sample sizes are large. Finally, the ability of these recursive algorithms to deal with very high-dimensional data is illustrated on the robust estimation of television audience profiles conditional on the total time spent watching television over a period of 24 hours.

This paper presents a new derivation of nonparametric distribution estimation with right-censored data. It is based on an extension of the predictive inferences to compound evidence. The estimate is recursive and exact, and no stochastic approximation is needed: it simply requires that the censored data are processed in decreasing order. Only in this case the recursion provides exact posterior predictive distributions for subsequent samples under a Dirichlet process prior. The resulting estimate is equivalent to the Susarla-VanRyzin estimator and to the beta-Stacy process.

The Stochastic Blockmodel [16] is a mixture model for heterogeneous network data. Unlike the usual statistical framework, new nodes give additional information about the previous ones in this model. Thereby the distribution of the degrees concentrates in points conditionally on the node class. We show under a mild assumption that classification, estimation and model selection can actually be achieved with no more than the empirical degree data. We provide an algorithm able to process very large networks and consistent estimators based on it. In particular, we prove a bound of the probability of misclassification of at least one node, including when the number of classes grows.

We focus on a capture-recapture model in which capture probabilities arise from an unspecified distribution $F$. We show that model parameters are identifiable based on the unconditional likelihood. This is not true with the conditional likelihood. We also clarify that consistency and asymptotic equivalence of maximum likelihood estimators based on conditional and unconditional likelihood do not hold. We show that estimates of the undetected fraction of population based on the unconditional likelihood converge to the so-called estimable sharpest lower bound and we derive a new asymptotic equivalence result. We finally provide theoretical and simulation arguments in favor of the use of the unconditional likelihood rather than the conditional likelihood especially when one is willing to infer on the sharpest lower bound.