We consider M estimation of a regression model with a nuisance parameter and a vector of other parameters. The unknown distribution of the residuals is not assumed to be normal or symmetric. Simple and easily estimated formulas are given for the dominant terms of the bias and skewness of the parameter estimates. For the linear model these are proportional to the skewness of the ‘independent’ variables. For a nonlinear model, its linear component plays the role of these independent variables, and a second term must be added proportional to the covariance of its linear and quadratic components. For the least squares estimate with normal errors this term was derived by Box [1]. We also consider the effect of a large number of parameters, and the case of random independent variables.

The paper establishes a functional central limit theorem for the empirical distribution function of a stationary, causal, ARMA process given by X_s,t=∑_i≥0∑_j≥0a_i,j ξ_s−i,t−j, (s,t)∈Z², where the ξ_i,j are independent and identically distributed, zero mean innovations. By judicious choice of σ−fields and element enumeration, one dimensional martingale arguments are employed to establish the result.

We consider a statistical experiment where two types of contaminated data are observed. Therein, both data sets are affected by additive measurement errors but the scaling factors of the error density may be different and/or the observations have been averaged over different numbers of independent replicates. That kind of heteroscedasticity of the data allows us to identify the target density although the error density is unknown and we can allow that the characteristic function of the error variables may have zeros. We introduce a novel nonparametric procedure which estimates the target density with nearly optimal convergence rates. The main goal in this paper is to derive the upper and lower bounds for the convergence rates. A small simulation study addresses the finite sample properties of the procedure.

Network engineering for quality-of-service (QoS) of Internet voice communication (VoIP) can benefit substantially from simulation study of the VoIP packet traffic on a network of routers. This requires accurate statistical models for the packet arrivals to the network from a gateway. The arrival point process is the superposition, or statistical multiplexing, of the arrival processes of packets of individual calls. The packets of each call form a transient point process with on-intervals of transmission and off-intervals of silence. This article presents the development and validation of models for the multiplexed process based on statistical analyses of VoIP traffic from the Global Crossing (GBLX) international network: 48 hr of VoIP arrival times and headers of 1.315 billion packets from 332018 calls. Statistical models and methods involve point processes and their superposition; time series autocorrelations and power spectra; long-range dependence; random effects and hierarchical modeling; bootstrapping; robust estimation; modeling independence and identical distribution; and visualization methods for model building. The result is two models validated by the analyses that can generate accurate synthetic multiplexed packet traffic. One is a semi-empirical model: empirical data are a part of the model. The second is a mathematical model: the components are parametric statistical models. This is the first comprehensive modeling of VoIP traffic based on data from a service provider carrying a full range of VoIP applications. The models can be used for simulation of any IP network architecture, wireline or wireless, because the modeling is for the IP-inbound traffic to an IP network. This is achieved because the GBLX data, collected on an IP link, are very close to their properties when they entered the GBLX network.

In this paper, we study the behavior of a kernel estimator of the regression function in the right censored model with α-mixing data. The uniform strong consistency over a real compact set of the estimate is established along with a rate of convergence. Some simulations are carried out to illustrate the behavior of the estimate with different examples for finite sample sizes.

We consider the problem of modeling heteroscedasticity in semiparametric regression analysis of cross-sectional data. Existing work in this setting is rather limited and mostly adopts a fully nonparametric variance structure. This approach is hampered by curse of dimensionality in practical applications. Moreover, the corresponding asymptotic theory is largely restricted to estimators that minimize certain smooth objective functions. The asymptotic derivation thus excludes semiparametric quantile regression models. To overcome these drawbacks, we study a general class of location-dispersion regression models, in which both the location function and the dispersion function are semiparametrically modeled. We establish unified asymptotic theory which is valid for many commonly used semiparametric structures such as the partially linear structure and single-index structure. We provide easy to check sufficient conditions and illustrate them through examples. Our theory permits non-smooth location or dispersion functions, thus allows for semiparametric quantile heteroscedastic regression and robust estimation in semiparametric mean regression. Simulation studies indicate significant efficiency gain in estimating the parametric component of the location function. The results are applied to analyzing a data set on gasoline consumption.

Suppose $n^{1/2}(\widehat{\theta}_{n}-\theta)\rightarrow \mathcal{N}_{p}(0,V(\theta))$ as n→∞ for some estimate $\widehat{\theta}_{n}$ of θ in R^p. If p=1 and g(θ)=∫₀^θV(x)^−1/2dx, it is well known that $n^{1/2}(g(\widehat{\theta}_{n})-g(\theta))\rightarrow \mathcal{N}(0,1)$ as n→∞, the distribution often being less skew so that inference based on the approximation $n^{1/2}(g(\widehat{\theta}_{n})-g(\theta))\sim \mathcal{N}(0,1)$should be more accurate than inference based on the approximation $V(\widehat{\theta}_{n})^{-1/2}n^{1/2}(\widehat{\theta}_{n}-\theta)\sim \mathcal{N}(0,1)$. If p>1 there is generally no such one to one transformation g(⋅). We consider three different types of stabilization of V(θ). We also consider the problem of finding g(⋅) so that the components of $g(\widehat{\theta}_{n})$ are asymptotically independent.

The purpose of this paper is to estimate the intensity of a Poisson process $N$ by using thresholding rules. In this paper, the intensity, defined as the derivative of the mean measure of $N$ with respect to $ndx$ where $n$ is a fixed parameter, is assumed to be non-compactly supported. The estimator $\tilde{f}_{n,\gamma}$ based on random thresholds is proved to achieve the same performance as the oracle estimator up to a possible logarithmic term. Then, minimax properties of $\tilde{f}_{n,\gamma}$ on Besov spaces $\mathcal{B}^{\alpha}_{p,q}$ are established. Under mild assumptions, we prove that $$\sup_{f\in \mathcal{B}^{\alpha}_{p,q}\cap \mathbb{L}_{\infty}}\mathbb{E}\|\tilde{f}_{n,\gamma}-f\|^{2}\leq C\left(\frac{ {\log \,}n}{n}\right)^{\frac{\alpha}{\alpha+\frac{1}{2}+\bigl(\frac{1}{2}-\frac{1}{p}\bigr)_{+}}}$$ and the lower bound of the minimax risk for $\mathcal{B}^{\alpha}_{p,q}\cap \mathbb{L}_{\infty}$ coincides with the previous upper bound up to the logarithmic term. This new result has two consequences. First, it establishes that the minimax rate of Besov spaces $\mathcal{B}^{\alpha}_{p,q}$ with $p≤2$ when non compactly supported functions are considered is the same as for compactly supported functions up to a logarithmic term. When $p>2$, the rate exponent, which depends on $p$, deteriorates when $p$ increases, which means that the support plays a harmful role in this case. Furthermore, $\tilde{f}_{n,\gamma}$ is adaptive minimax up to a logarithmic term. Our procedure is based on data-driven thresholds. As usual, they depend on a tuning parameter $γ$ whose optimal value is hard to estimate from the data. In this paper, we study the problem of calibrating $γ$ both theoretically and practically. Finally, some simulations are provided, proving the excellent practical behavior of our procedure with respect to the support issue.

We study the posterior distribution of the Bayesian multiple change-point regression problem when the number and the locations of the change-points are unknown. While it is relatively easy to apply the general theory to obtain the $O(1/\sqrt{n})$ rate up to some logarithmic factor, showing the parametric rate of convergence of the posterior distribution requires additional work and assumptions. Additionally, we demonstrate the asymptotic normality of the segment levels under these assumptions. For inferences on the number of change-points, we show that the Bayesian approach can produce a consistent posterior estimate. Finally, we show that consistent posterior for model selection necessarily implies that the parametric rate for posterior estimation stated previously cannot be uniform over the class of models we consider. This is the Bayesian version of the same phenomenon that has been noted and studied by other authors.

We present theoretical properties of the log-concave maximum likelihood estimator of a density based on an independent and identically distributed sample in ℝ^d. Our study covers both the case where the true underlying density is log-concave, and where this model is misspecified. We begin by showing that for a sequence of log-concave densities, convergence in distribution implies much stronger types of convergence – in particular, it implies convergence in Hellinger distance and even in certain exponentially weighted total variation norms. In our main result, we prove the existence and uniqueness of a log-concave density that minimises the Kullback–Leibler divergence from the true density over the class of all log-concave densities, and also show that the log-concave maximum likelihood estimator converges almost surely in these exponentially weighted total variation norms to this minimiser. In the case of a correctly specified model, this demonstrates a strong type of consistency for the estimator; in a misspecified model, it shows that the estimator converges to the log-concave density that is closest in the Kullback–Leibler sense to the true density.

Multiple hypothesis testing often encounters composite nulls and intractable alternative distributions. In this case, using p-values that are defined as maximum significance levels over all null distributions (“p_max”) often leads to very conservative testing. We propose constructing p-values via maximization under linear constraints imposed by data’s empirical distribution, and show that these p-values allow the false discovery rate (FDR) to be controlled with substantially more power than p_max.

Consider a linear regression model with independent and identically normally distributed random errors. Suppose that the parameter of interest is a specified linear combination of the regression parameters. We prove that the usual confidence interval for this parameter is admissible within a broad class of confidence intervals.

We consider a Bayesian hierarchical version of the normal theory general linear model which is practically relevant in the sense that it is general enough to have many applications and it is not straightforward to sample directly from the corresponding posterior distribution. Thus we study a block Gibbs sampler that has the posterior as its invariant distribution. In particular, we establish that the Gibbs sampler converges at a geometric rate. This allows us to establish conditions for a central limit theorem for the ergodic averages used to estimate features of the posterior. Geometric ergodicity is also a key requirement for using batch means methods to consistently estimate the variance of the asymptotic normal distribution. Together, our results give practitioners the tools to be as confident in inferences based on the observations from the Gibbs sampler as they would be with inferences based on random samples from the posterior. Our theoretical results are illustrated with an application to data on the cost of health plans issued by health maintenance organizations.

Confidence intervals based on penalized maximum likelihood estimators such as the LASSO, adaptive LASSO, and hard-thresholding are analyzed. In the known-variance case, the finite-sample coverage properties of such intervals are determined and it is shown that symmetric intervals are the shortest. The length of the shortest intervals based on the hard-thresholding estimator is larger than the length of the shortest interval based on the adaptive LASSO, which is larger than the length of the shortest interval based on the LASSO, which in turn is larger than the standard interval based on the maximum likelihood estimator. In the case where the penalized estimators are tuned to possess the ‘sparsity property’, the intervals based on these estimators are larger than the standard interval by an order of magnitude. Furthermore, a simple asymptotic confidence interval construction in the ‘sparse’ case, that also applies to the smoothly clipped absolute deviation estimator, is discussed. The results for the known-variance case are shown to carry over to the unknown-variance case in an appropriate asymptotic sense.

The Importance Sampling method is used as an alternative approach to MCMC in repeated Bayesian estimations. In the particular context of numerous data sets, MCMC algorithms have to be called on several times which may become computationally expensive. Since Importance Sampling requires a sample from a posterior distribution, our idea is to use MCMC to generate only a certain number of Markov chains and use them later in the subsequent IS estimations. For each Importance Sampling procedure, the suitable chain is selected by one of three criteria we present here. The first and second criteria are based on the L¹ norm of the difference between two posterior distributions and their Kullback-Leibler divergence respectively. The third criterion results from minimizing the variance of IS estimate. A supplementary automatic selection procedure is also proposed to choose those posterior for which Markov chains will be generated and to avoid arbitrary choice of importance functions. The featured methods are illustrated in simulation studies on three types of Poisson model: simple Poisson model, Poisson regression model and Poisson regression model with extra Poisson variability. Different parameter settings are considered.

Most of the non-asymptotic theoretical work in regression is carried out for the square loss, where estimators can be obtained through closed-form expressions. In this paper, we use and extend tools from the convex optimization literature, namely self-concordant functions, to provide simple extensions of theoretical results for the square loss to the logistic loss. We apply the extension techniques to logistic regression with regularization by the ℓ₂-norm and regularization by the ℓ₁-norm, showing that new results for binary classification through logistic regression can be easily derived from corresponding results for least-squares regression.

In this paper, we propose a fast image deconvolution algorithm that combines adaptive block thresholding and Vaguelet-Wavelet Decomposition. The approach consists in first denoising the observed image using a wavelet-domain Stein block thresholding, and then inverting the convolution operator in the Fourier domain. Our main theoretical result investigates the minimax rates over Besov smoothness spaces, and shows that our block estimator can achieve the optimal minimax rate, or is at least nearly-minimax in the least favorable situation. The resulting algorithm is simple to implement and fast. Its computational complexity is dominated by that of the FFT. We report a simulation study to support our theoretical findings. The practical performance of our block vaguelet-wavelet deconvolution compares very favorably to existing competitors on a large set of test images.

In regression with a high-dimensional predictor vector, dimension reduction methods aim at replacing the predictor by a lower dimensional version without loss of information on the regression. In this context, the so-called central mean subspace is the key of dimension reduction. The last two decades have seen the emergence of many methods to estimate the central mean subspace. In this paper, we go one step further, and we study the performances of a k-nearest neighbor type estimate of the regression function, based on an estimator of the central mean subspace. In our setting, the predictor lies in ℝ^p with fixed p, i.e. it does not depend on the sample size. The estimate is first proved to be consistent. Improvement due to the dimension reduction step is then observed in term of its rate of convergence. All the results are distributions-free. As an application, we give an explicit rate of convergence using the SIR method. The method is illustrated by a simulation study.

Suitable sequentially rejective multiple test procedures allow to “zoom in" on clusters of relevant variables in high-dimensional regression (Meinshausen [7]), or on regions of interest in some search space (Heinrich et al. [3]; Meinshausen et al. [8]). As a common framework for these schemes we propose to consider multiple testing along a tree of hypotheses together with a “keep rejecting until first acceptance" rule. Particular topics addressed in this note are control of the familywise error, and some variants and basic properties of the procedure.

A classical approach for dealing with a multiple testing problem is to restrict attention to procedures that control the familywise error rate (FWER), the probability of at least one false rejection. In many applications, one might be willing to tolerate more than one false rejection provided the number of such cases is controlled, thereby increasing the ability of a procedure to detect false null hypotheses. This suggests replacing control of the FWER by controlling the probability of k or more false rejections, which is called the k-FWER. In this article, a unified approach is presented for deriving the k-FWER controlling procedures. We first generalize the well-known closure principle in the context of the FWER to the case of controlling the k-FWER. Then, we discuss how to derive the k-FWER controlling stepup procedures based on marginal p-values using this principle. We show that, under certain conditions, generalized closed testing procedures can be reduced to stepup procedures, and any stepup procedure is equivalent to a generalized closed testing procedure. Finally, we generalize the well-known Hommel procedure in two directions, and show that any generalized Hommel procedure is equivalent to a generalized closed testing procedure with the same critical values.

In this article we introduce a multivariate structural linear error-in-variables model which is suitable for longitudinal data. We construct estimators of the regression parameters, which correspond to the modified least squares estimators used in the univariate case. We show that these estimators are consistent. We prove a central limit theorem, which is completely data-based, under the assumption that the vector of latent variables belongs to the generalized domain of attraction of the normal law. Our results can be viewed as an extension of the results of [12] to include the longitudinal case.

We consider drift estimation of a discretely observed Ornstein-Uhlenbeck process driven by a possibly heavy-tailed symmetric Lévy process with positive activity index β. Under an infill and large-time sampling design, we first establish an asymptotic normality of a self-weighted least absolute deviation estimator with the rate of convergence being $\sqrt{n}h_{n}^{1-1/\beta}$, where n denotes sample size and h_n>0 the sampling mesh satisfying that h_n→0 and nh_n→∞. This implies that the rate of convergence is determined by the most active part of the driving Lévy process; the presence of a driving Wiener part leads to $\sqrt{nh_{n}}$, which is familiar in the context of asymptotically efficient estimation of diffusions with compound Poisson jumps, while a pure-jump driving Lévy process leads to a faster one. Also discussed is how to construct corresponding asymptotic confidence regions without full specification of the driving Lévy process. Second, by means of a polynomial type large deviation inequality we derive convergence of moments of our estimator under additional conditions.

Temporal text data such as news feeds cannot be adequately modeled by standard n-grams which correspond to multinomial or Markov chain models. Instead, we examine the application of local n-grams to modeling time stamped documents. We derive the asymptotic bias and variance and consider the bandwidth selection problem. Experimental results are presented on news feeds and web search query logs.

We propose a family of statistical models for social network evolution over time, which represents an extension of Exponential Random Graph Models (ERGMs). Many of the methods for ERGMs are readily adapted for these models, including maximum likelihood estimation algorithms. We discuss models of this type and their properties, and give examples, as well as a demonstration of their use for hypothesis testing and classification. We believe our temporal ERG models represent a useful new framework for modeling time-evolving social networks, and rewiring networks from other domains such as gene regulation circuitry, and communication networks.

Frailty models derived from the proportional hazards regression model are frequently used to analyze clustered right-censored survival data. We propose a semiparametric Bayesian methodology for this purpose, modeling both the unknown baseline hazard and density of the random effects using mixtures of B-splines. The posterior distributions for all regression coefficients and spline parameters are obtained using Markov Chain Monte Carlo (MCMC). The methodology permits the use of weighted mixtures of parametric and nonparametric components in modeling the hazard function and frailty distribution; in addition, the spline knots may also be selected adaptively using reversible-jump MCMC. Simulations indicate that the method produces smooth and accurate posterior hazard and frailty density estimates. The Bayesian approach not only produces point estimators that outperform existing approaches in certain circumstances, but also offers a wealth of information about the parameters of interest in the form of MCMC samples from the joint posterior probability distribution. We illustrate the adaptability of the method with data from a study of congestive heart failure.

When testing a null hypothesis H₀: θ=θ₀ in a Bayesian framework, the Savage–Dickey ratio (Dickey, 1971) is known as a specific representation of the Bayes factor (O’Hagan and Forster, 2004) that only uses the posterior distribution under the alternative hypothesis at θ₀, thus allowing for a plug-in version of this quantity. We demonstrate here that the Savage–Dickey representation is in fact a generic representation of the Bayes factor and that it fundamentally relies on specific measure-theoretic versions of the densities involved in the ratio, instead of being a special identity imposing some mathematically void constraints on the prior distributions. We completely clarify the measure-theoretic foundations of the Savage–Dickey representation as well as of the later generalisation of Verdinelli and Wasserman (1995). We provide furthermore a general framework that produces a converging approximation of the Bayes factor that is unrelated with the approach of Verdinelli and Wasserman (1995) and propose a comparison of this new approximation with their version, as well as with bridge sampling and Chib’s approaches.

In this work we give our contribution to the problem of segmentation with plug-in procedures. We propose general sufficient conditions under which plug in procedure are efficient. We also give an algorithm that satisfy these conditions. We apply this algorithm to hyperspectral images segmentation. Hyperspectral images are images that have both spatial and spectral coherence with thousands of spectral bands on each pixel. In the proposed procedure we combine a reduction dimension technique and a spatial regularization technique. This regularization is based on the mixlet modeling of Kolaczyck et al. [10].

This paper presents asymptotic properties of the maximum pseudo-likelihood estimator of a vector parameterizing a stationary Gibbs point process. Sufficient conditions, expressed in terms of the local energy function defining a Gibbs point process, to establish strong consistency and asymptotic normality results of this estimator depending on a single realization, are presented. These results are general enough to no longer require the local stability and the linearity in terms of the parameters of the local energy function. We consider characteristic examples of such models, the Lennard-Jones and the finite range Lennard-Jones models. We show that the different assumptions ensuring the consistency are satisfied for both models whereas the assumptions ensuring the asymptotic normality are fulfilled only for the finite range Lennard-Jones model.

We consider the problem of multivariate density estimation when the unknown density is assumed to follow a particular form of dimensionality reduction, a noisy independent factor analysis (IFA) model. In this model the data are generated by a number of latent independent components having unknown distributions and are observed in Gaussian noise. We do not assume that either the number of components or the matrix mixing the components are known. We show that the densities of this form can be estimated with a fast rate. Using the mirror averaging aggregation algorithm, we construct a density estimator which achieves a nearly parametric rate $(\log^{1/4}{n})/\sqrt{n}$, independent of the dimensionality of the data, as the sample size n tends to infinity. This estimator is adaptive to the number of components, their distributions and the mixing matrix. We then apply this density estimator to construct nonparametric plug-in classifiers and show that they achieve the best obtainable rate of the excess Bayes risk, to within a logarithmic factor independent of the dimension of the data. Applications of this classifier to simulated data sets and to real data from a remote sensing experiment show promising results.

In this article, we establish the asymptotic normality of the posterior distribution for the natural parameter in an exponential family based on independent and identically distributed data. The mode of convergence is expected Kullback-Leibler distance and the number of parameters p is increasing with the sample size n. Using this, we give an asymptotic expansion of the Shannon mutual information valid when p=p_n increases at a sufficiently slow rate. The second term in the asymptotic expansion is the largest term that depends on the prior and can be optimized to give Jeffreys’ prior as the reference prior in the absence of nuisance parameters. In the presence of nuisance parameters, we find an analogous result for each fixed value of the nuisance parameter. In three examples, we determine the rates at which p_n can be allowed to increase while still retaining asymptotic normality and the reference prior property.

We consider the models Y_i,n=∫₀^i/nσ(s)dW_s+τ(i/n)ε_i,n, and Ỹ_i,n=σ(i/n)W_i/n+τ(i/n)ε_i,n, i=1,…,n, where (W_t)_t∈[0,1] denotes a standard Brownian motion and ε_i,n are centered i.i.d. random variables with E (ε_i,n²)=1 and finite fourth moment. Furthermore, σ and τ are unknown deterministic functions and (W_t)_t∈[0,1] and (ε_1,n,…,ε_n,n) are assumed to be independent processes. Based on a spectral decomposition of the covariance structures we derive series estimators for σ² and τ² and investigate their rate of convergence of the MISE in dependence of their smoothness. To this end specific basis functions and their corresponding Sobolev ellipsoids are introduced and we show that our estimators are optimal in minimax sense. Our work is motivated by microstructure noise models. A major finding is that the microstructure noise ε_i,n introduces an additionally degree of ill-posedness of 1/2; irrespectively of the tail behavior of ε_i,n. The performance of the estimates is illustrated by a small numerical study.

We propose a model selection approach for covariance estimation of a stochastic process. Under very general assumptions, observing i.i.d replications of the process at fixed observation points, we construct an estimator of the covariance function by expanding the process onto a collection of basis functions. We study the non asymptotic property of this estimate and give a tractable way of selecting the best estimator among a possible set of candidates. The optimality of the procedure is proved via an oracle inequality which warrants that the best model is selected.

We prove that the Hodrick-Prescott Filter (HPF), a commonly used method for smoothing econometric time series, is a special case of a linear penalized spline model with knots placed at all observed time points (except the first and last) and uncorrelated residuals. This equivalence then furnishes a rich variety of existing data-driven parameter estimation methods, particularly restricted maximum likelihood (REML) and generalized cross-validation (GCV). This has profound implications for users of HPF who have hitherto typically relied on subjective choice, rather than estimation, for the smoothing parameter. By viewing estimates as roots of an appropriate quadratic estimating equation, we also present a new approach for constructing confidence intervals for the smoothing parameter. The method is akin to a parametric bootstrap where Monte Carlo simulation is replaced by saddlepoint approximation, and provides a fast and accurate alternative to exact methods, when they exist, e.g. REML. More importantly, it is also the only computationally feasible method when no other methods, exact or otherwise, exist, e.g. GCV. The methodology is demonstrated on the Gross National Product (GNP) series originally analyzed by Hodrick and Prescott (1997). With proper attention paid to residual correlation structure, we show that REML-based estimation delivers an appropriate smooth for both the GNP series and its returns.

In this article we study the estimation of the location of jump points in the first derivative (referred to as kinks) of a regression function μ in two random design models with different long-range dependent (LRD) structures. The method is based on the zero-crossing technique and makes use of high-order kernels. The rate of convergence of the estimator is contingent on the level of dependence and the smoothness of the regression function μ. In one of the models, the convergence rate is the same as the minimax rate for kink estimation in the fixed design scenario with i.i.d. errors which suggests that the method is optimal in the minimax sense.

Supervised dimension reduction has proven effective in analyzing data with complex structure. The primary goal is to seek the reduced subspace of minimal dimension which is sufficient for summarizing the data structure of interest. This paper investigates the supervised dimension reduction in high dimensional classification context, and proposes a novel method for estimating the dimension reduction subspace while retaining the ideal classification boundary based on the original dataset. The proposed method combines the techniques of margin based classification and shrinkage estimation, and can estimate the dimension and the directions of the reduced subspace simultaneously. Both theoretical and numerical results indicate that the proposed method is highly competitive against its competitors, especially when the dimension of the covariates exceeds the sample size.

We consider a Bayesian approach to model selection in Gaussian linear regression, where the number of predictors might be much larger than the number of observations. From a frequentist view, the proposed procedure results in the penalized least squares estimation with a complexity penalty associated with a prior on the model size. We investigate the optimality properties of the resulting model selector. We establish the oracle inequality and specify conditions on the prior that imply its asymptotic minimaxity within a wide range of sparse and dense settings for “nearly-orthogonal” and “multicollinear” designs.

A lattice rule with a randomly-shifted lattice estimates a mathematical expectation, written as an integral over the s-dimensional unit hypercube, by the average of n evaluations of the integrand, at the n points of the shifted lattice that lie inside the unit hypercube. This average provides an unbiased estimator of the integral and, under appropriate smoothness conditions on the integrand, it has been shown to converge faster as a function of n than the average at n independent random points (the standard Monte Carlo estimator). In this paper, we study the behavior of the estimation error as a function of the random shift, as well as its distribution for a random shift, under various settings. While it is well known that the Monte Carlo estimator obeys a central limit theorem when n→∞, the randomized lattice rule does not, due to the strong dependence between the function evaluations. We show that for the simple case of one-dimensional integrands, the limiting error distribution is uniform over a bounded interval if the integrand is non-periodic, and has a square root form over a bounded interval if the integrand is periodic. We find that in higher dimensions, there is little hope to precisely characterize the limiting distribution in a useful way for computing confidence intervals in the general case. We nevertheless examine how this error behaves as a function of the random shift from different perspectives and on various examples. We also point out a situation where a classical central-limit theorem holds when the dimension goes to infinity, we provide guidelines on when the error distribution should not be too far from normal, and we examine how far from normal is the error distribution in examples inspired from real-life applications.

This paper considers the problem of adaptive estimation of a template in a randomly shifted curve model. Using the Fourier transform of the data, we show that this problem can be transformed into a linear inverse problem with a random operator. Our aim is to approach the estimator that has the smallest risk on the true template over a finite set of linear estimators defined in the Fourier domain. Based on the principle of unbiased empirical risk minimization, we derive a nonasymptotic oracle inequality in the case where the law of the random shifts is known. This inequality can then be used to obtain adaptive results on Sobolev spaces as the number of observed curves tend to infinity. Some numerical experiments are given to illustrate the performances of our approach.

We introduce models for the analysis of functional data observed at multiple time points. The dynamic behavior of functional data is decomposed into a time-dependent population average, baseline (or static) subject-specific variability, longitudinal (or dynamic) subject-specific variability, subject-visit-specific variability and measurement error. The model can be viewed as the functional analog of the classical longitudinal mixed effects model where random effects are replaced by random processes. Methods have wide applicability and are computationally feasible for moderate and large data sets. Computational feasibility is assured by using principal component bases for the functional processes. The methodology is motivated by and applied to a diffusion tensor imaging (DTI) study designed to analyze differences and changes in brain connectivity in healthy volunteers and multiple sclerosis (MS) patients. An R implementation is provided.

This paper studies a generic sparse regression problem with a customizable sparsity pattern matrix, motivated by, but not limited to, a supervised gene clustering problem in microarray data analysis. The clustered lasso method is proposed with the l₁-type penalties imposed on both the coefficients and their pairwise differences. Somewhat surprisingly, it behaves differently than the lasso or the fused lasso – the exact clustering effect expected from the l₁ penalization is rarely seen in applications. An asymptotic study is performed to investigate the power and limitations of the l₁-penalty in sparse regression. We propose to combine data-augmentation and weights to improve the l₁ technique. To address the computational issues in high dimensions, we successfully generalize a popular iterative algorithm both in practice and in theory and propose an ‘annealing’ algorithm applicable to generic sparse regressions (including the fused/clustered lasso). Some effective accelerating techniques are further investigated to boost the convergence. The accelerated annealing (AA) algorithm, involving only matrix multiplications and thresholdings, can handle a large design matrix as well as a large sparsity pattern matrix.

A general iterative algorithm is developed for the computation of reduced-bias parameter estimates in regular statistical models through adjustments to the score function. The algorithm unifies and provides appealing new interpretation for iterative methods that have been published previously for some specific model classes. The new algorithm can usefully be viewed as a series of iterative bias corrections, thus facilitating the adjusted score approach to bias reduction in any model for which the first-order bias of the maximum likelihood estimator has already been derived. The method is tested by application to a logit-linear multiple regression model with beta-distributed responses; the results confirm the effectiveness of the new algorithm, and also reveal some important errors in the existing literature on beta regression.

This paper studies the estimation of a large covariance matrix. We introduce a novel procedure called ChoSelect based on the Cholesky factor of the inverse covariance. This method uses a dimension reduction strategy by selecting the pattern of zero of the Cholesky factor. Alternatively, ChoSelect can be interpreted as a graph estimation procedure for directed Gaussian graphical models. Our approach is particularly relevant when the variables under study have a natural ordering (e.g. time series) or more generally when the Cholesky factor is approximately sparse. ChoSelect achieves non-asymptotic oracle inequalities with respect to the Kullback-Leibler entropy. Moreover, it satisfies various adaptive properties from a minimax point of view. We also introduce and study a two-stage procedure that combines ChoSelect with the Lasso. This last method enables the practitioner to choose his own trade-off between statistical efficiency and computational complexity. Moreover, it is consistent under weaker assumptions than the Lasso. The practical performances of the different procedures are assessed on numerical examples.

In Geographical Information Systems, spatial point pattern data are often analysed by dividing space into pixels, recording the presence or absence of points in each pixel, and fitting a logistic regression. We study weaknesses of this approach, propose improvements, and demonstrate an application to prospective geology in Western Australia. Models based on different pixel grids are incompatible (a ‘change-of-support’ problem) unless the pixels are very small. On a fine pixel grid, a spatial logistic regression is approximately a Poisson point process with loglinear intensity; we give explicit distributional bounds. For a loglinear Poisson process, the optimal parameter estimator from pixel data is not spatial logistic regression, but complementary log-log regression with an offset depending on pixel area. If the pixel raster is randomly subsampled, logistic regression is conditionally optimal. Bias and efficiency depend strongly on the spatial regularity of the covariates. For discontinuous covariates, we propose a new algorithmic strategy in which pixels are subdivided, and demonstrate its efficiency.

A popular approach to smooth models for longitudinal data is to express the model as a mixed model, since this often leads to immediate model fitting with standard procedures. This approach is particularly appealing when truncated polynomials are used as a basis for the smoothing, as the mixed model representation is almost immediate. We show that this approach can lead to a severely biased estimate of the overall population effect and to confidence intervals with undesirable properties. We use penalization to investigate an alternative approach with either B-spline or truncated polynomial bases and show that this new approach does not suffer from the same defects. Our models are defined in terms of B-splines or truncated polynomials with appropriate penalties, but can be expressed as mixed models; this also gives access to fitting with standard procedures. We illustrate our methods with an analysis of two data sets: (a) a balanced data set on Canadian weather and (b) an unbalanced data set on the growth of children.

We study convergence rates of Bayesian density estimators based on finite location-scale mixtures of exponential power distributions. We construct approximations of β-Hölder densities be continuous mixtures of exponential power distributions, leading to approximations of the β-Hölder densities by finite mixtures. These results are then used to derive posterior concentration rates, with priors based on these mixture models. The rates are minimax (up to a logn term) and since the priors are independent of the smoothness the rates are adaptive to the smoothness.

The use of penalization, or regularization, has become common in high-dimensional statistical analysis, where an increasingly frequent goal is to simultaneously select important variables and estimate their effects. It has been shown by several authors that these goals can be achieved by minimizing some parameter-dependent “goodness-of-fit” function (e.g., a negative loglikelihood) subject to a penalization that promotes sparsity. Penalty functions that are singular at the origin have received substantial attention, arguably beginning with the Lasso penalty [62].

The current literature tends to focus on specific combinations of differentiable goodness-of-fit functions and penalty functions singular at the origin. One result of this combined specificity has been a proliferation in the number of computational algorithms designed to solve fairly narrow classes of optimization problems involving objective functions that are not everywhere continuously differentiable. In this paper, we propose a general class of algorithms for optimizing an extensive variety of nonsmoothly penalized objective functions that satisfy certain regularity conditions. The proposed framework utilizes the majorization-minimization (MM) algorithm as its core optimization engine. In the case of penalized regression models, the resulting algorithms employ iterated soft-thresholding, implemented componentwise, allowing for fast and stable updating that avoids the need for inverting high-dimensional matrices. We establish convergence theory under weaker assumptions than previously considered in the statistical literature. We also demonstrate the exceptional effectiveness of new acceleration methods, originally proposed for the EM algorithm, in this class of problems. Simulation results and a microarray data example are provided to demonstrate the algorithm’s capabilities and versatility.

We consider high frequency observations of a semi-martingale. From these data, we build simple test statistics allowing to distinguish between the two following situations: i) the data generating process is an Itō semi-martingale; ii) the data generating process is a Multifractal Random Walk. We also investigate the finite sample behavior of the test statistics on some simulated data.

Let $(Y_i,Z_i)_{i≥1}$ be a sequence of independent, identically distributed (i.i.d.) random vectors taking values in $ℝ^k×ℝ^d$, for some integers $k$ and $d$. Given $z∈ℝ^d$, we provide a nonstandard functional limit law for the sequence of functional increments of the compound empirical process, namely $$\mathbf{\Delta}_{n,\mathfrak{c}}(h_{n},z,\cdot):=\frac{1}{nh_{n}}\sum_{i=1}^{n}1_{[0,\cdot)}\Big(\frac{Z_{i}-z}{{h_{n}}^{1/d}}\Big)Y_{i}.$$ Provided that $nh_n∼c\log n$ as $n→∞$, we obtain, under some natural conditions on the conditional exponential moments of $Y\mid Z=z$, that $$\mathbf{\Delta}_{n,\mathfrak{c}}(h_{n},z,\cdot)\leadsto \Gamma \text{ almost surely},$$ where $↝$ denotes the clustering process under the sup norm on $[0,1)^d$. Here, $Γ$ is a compact set that is related to the large deviations of certain compound Poisson processes.

We propose a second-order accurate method to estimate the eigenvectors of extremely large matrices thereby addressing a problem of relevance to statisticians working in the analysis of very large datasets. More specifically, we show that averaging eigenvectors of randomly subsampled matrices efficiently approximates the true eigenvectors of the original matrix under certain conditions on the incoherence of the spectral decomposition. This incoherence assumption is typically milder than those made in matrix completion and allows eigenvectors to be sparse. We discuss applications to spectral methods in dimensionality reduction and information retrieval.

In general the empirical likelihood method can improve the performance of estimators by including additional information about the underlying data distribution. Application of the method to kernel density estimation based on independent and identically distributed data always improves the estimation in second order. In this paper we consider estimation of the error density in nonparametric regression by residual-based kernel estimation. We investigate whether the estimator is improved when additional information is included by the empirical likelihood method. We show that the convergence rate is not effected, but in comparison to the residual-based kernel estimator we observe a change in the asymptotic bias of the empirical likelihood estimator in first order and in the asymptotic variance in second order. Those changes do not result in a general uniform improvement of the estimation, but in typical examples we demonstrate the good performance of the residual-based empirical likelihood estimator in asymptotic theory as well as in simulations.

In Fernández-Durán [4], a new family of circular distributions based on nonnegative trigonometric sums (NNTS models) is developed. Because the parameter space of this family is the surface of the hypersphere, an efficient Newton-like algorithm on manifolds is generated in order to obtain the maximum likelihood estimates of the parameters.

This work was motivated by Cox and O’Sullivan (1990) who derived the optimal convergence rates for smoothing spline estimates when the loss function is sufficiently smooth. However, the study of statistical estimates resulting from nonsmooth criteria functions has become popular in recent years. In this paper, we will study the asymptotic properties of the smoothing spline estimates when the criteria functions are insufficiently smooth. Here, the smoothing spline estimate is defined as an approximate solution to an M-estimating equation. We prove that if the derivative of loss function is Lipschitz, then the convergence rate and Bahadur type representation of the estimate can be derived simultaneously. For a specific class of loss functions with discontinuous derivatives, the Bahadur type representation is also presented provided that we know the convergence rate. Examples are given when Huber’s robust loss and median loss are employed.

We consider a two dimensional stochastic process (X,Y), which may have jump components and is not necessarily ergodic. There is an unknown parameter θ within the coefficients of (X,Y). The aim of this paper is to estimate θ from a regularly spaced sample of the process (X,Y). When the dynamic of X is known, an estimator is constructed by using a moment-based method. We show that our estimators will work if the Blumenthal-Getoor index of the jump part of Y is less than 2. What is perhaps the most interesting is the rate at which the estimators converge: it is $1/{\sqrt{n}}$ (as when the underlying processes are not contaminated by jumps) when that index is not greater than 1. When the dynamic of X is unknown, we introduce a spot volatility estimator-based approach to estimate θ. This approach can work even if the sample is contaminated by microstructure noise.

We propose an efficient solution to the problem of direct sampling from a hyper-inverse Wishart distribution in non-decomposable graphs. The method relies on local computations based on the standard junction tree representation of graphs and distribution theoretical results of constraint Wishart matrices.

We study the problem of detection of a p-dimensional sparse vector of parameters in the linear regression model with Gaussian noise. We establish the detection boundary, i.e., the necessary and sufficient conditions for the possibility of successful detection as both the sample size n and the dimension p tend to infinity. Testing procedures that achieve this boundary are also exhibited. Our results encompass the high-dimensional setting (p≫n). The main message is that, under some conditions, the detection boundary phenomenon that has been previously established for the Gaussian sequence model, extends to high-dimensional linear regression. Finally, we establish the detection boundaries when the variance of the noise is unknown. Interestingly, the rate of the detection boundary in high-dimensional setting with unknown variance can be different from the rate for the case of known variance.

Microarray experiments often yield a normal data matrix X whose rows correspond to genes and columns to samples. We commonly calculate test statistics Z=Xw, where Z_i is a test statistic for the ith gene, and apply false discovery rate (FDR) controlling methods to find interesting genes. For example, Z could measure the difference in expression levels between treatment and control groups and we could seek differentially expressed genes. The empirical cdf of Z is important for FDR methods, since its mean and variance determine the bias and variance of FDR estimates. Efron (2009b) has shown that if the columns of X are independent, the variance of the empirical cdf of Z only depends on the mean-squared row correlation.

Microarray data, however, frequently shows signs of column dependence. In this paper, we show that Efron’s result still holds under column dependence, and give a conservative (upwardly biased) estimator for the mean-squared row correlation. We show Fisher’s transformation for sample correlations is still normalizing and variance stabilizing under column dependence, and use it to construct a permutation-invariant test of column independence. Finally, we argue that estimating the mean-squared row correlation under column dependence is impossible in general. Code to perform our test is available in the R package “colcor,” available on CRAN.