Electronic Journal of Statistics Articles (Project Euclid)

The bias and skewness of M -estimators in regression

Thu, 05 Aug 2010 15:41 EDT

Christopher Withers, Saralees Nadarajah

Source: Electron. J. Statist., Volume 4, 1--14.

Abstract:
We consider M estimation of a regression model with a nuisance parameter and a vector of other parameters. The unknown distribution of the residuals is not assumed to be normal or symmetric. Simple and easily estimated formulas are given for the dominant terms of the bias and skewness of the parameter estimates. For the linear model these are proportional to the skewness of the ‘independent’ variables. For a nonlinear model, its linear component plays the role of these independent variables, and a second term must be added proportional to the covariance of its linear and quadratic components. For the least squares estimate with normal errors this term was derived by Box [1]. We also consider the effect of a large number of parameters, and the case of random independent variables.

Significance testing in quantile regression

Thu, 24 Jan 2013 10:34 EST

Stanislav Volgushev, Melanie Birke, Holger Dette, Natalie Neumeyer

Source: Electron. J. Statist., Volume 7, 105--145.

Abstract:
We consider the problem of testing significance of predictors in multivariate nonparametric quantile regression. A stochastic process is proposed, which is based on a comparison of the responses with a nonparametric quantile regression estimate under the null hypothesis. It is demonstrated that under the null hypothesis this process converges weakly to a centered Gaussian process and the asymptotic properties of the test under fixed and local alternatives are also discussed. In particular we show, that - in contrast to the nonparametric approach based on estimation of $L^{2}$-distances - the new test is able to detect local alternatives which converge to the null hypothesis with any rate $a_{n}\to 0$ such that $a_{n}\sqrt{n}\to\infty$ (here $n$ denotes the sample size). We also present a small simulation study illustrating the finite sample properties of a bootstrap version of the corresponding Kolmogorov-Smirnov test.

Minimax testing of a composite null hypothesis defined via a quadratic functional in the model of regression

Thu, 24 Jan 2013 10:34 EST

Laëtitia Comminges, Arnak S. Dalalyan

Source: Electron. J. Statist., Volume 7, 146--190.

Abstract:
We consider the problem of testing a particular type of composite null hypothesis under a nonparametric multivariate regression model. For a given quadratic functional $Q$, the null hypothesis states that the regression function $f$ satisfies the constraint $Q[f]=0$, while the alternative corresponds to the functions for which $Q[f]$ is bounded away from zero. On the one hand, we provide minimax rates of testing and the exact separation constants, along with a sharp-optimal testing procedure, for diagonal and nonnegative quadratic functionals. We consider smoothness classes of ellipsoidal form and check that our conditions are fulfilled in the particular case of ellipsoids corresponding to anisotropic Sobolev classes. In this case, we present a closed form of the minimax rate and the separation constant. On the other hand, minimax rates for quadratic functionals which are neither positive nor negative makes appear two different regimes: “regular” and “irregular”. In the “regular" case, the minimax rate is equal to $n^{-1/4}$ while in the “irregular” case, the rate depends on the smoothness class and is slower than in the “regular” case. We apply this to the problem of testing the equality of Sobolev norms of two functions observed in noisy environments.

On rate optimal local estimation in functional linear regression

Thu, 24 Jan 2013 10:34 EST

Jan Johannes, Rudolf Schenk

Source: Electron. J. Statist., Volume 7, 191--216.

Abstract:
We consider the estimation of the value of a linear functional of the slope parameter in functional linear regression, where scalar responses are modeled in dependence of random functions. The theory in this paper covers in particular point-wise estimation as well as the estimation of weighted averages of the slope parameter. We propose a plug-in estimator which is based on a dimension reduction technique and additional thresholding. It is shown that this estimator is consistent under mild assumptions. We derive a lower bound for the maximal mean squared error of any estimator over a certain ellipsoid of slope parameters and a certain class of covariance operators associated with the regressor. It is shown that the proposed estimator attains this lower bound up to a constant and hence it is minimax optimal. Our results are appropriate to discuss a wide range of possible regressors, slope parameters and functionals. They are illustrated by considering the point-wise estimation of the slope parameter or its derivatives and its average value over a given interval.

Semiparametric Bernstein–von Mises for the error standard deviation

Thu, 24 Jan 2013 10:34 EST

René de Jonge, Harry van Zanten

Source: Electron. J. Statist., Volume 7, 217--243.

Abstract:
We study Bayes procedures for nonparametric regression problems with Gaussian errors, giving conditions under which a Bernstein–von Mises result holds for the marginal posterior distribution of the error standard deviation. We apply our general results to show that a single Bayes procedure using a hierarchical spline-based prior on the regression function and an independent prior on the error variance, can simultaneously achieve adaptive, rate-optimal estimation of a smooth, multivariate regression function and efficient, $\sqrt{n}$-consistent estimation of the error standard deviation.

Identifiability of linear mixed effects models

Thu, 24 Jan 2013 10:34 EST

Wei Wang

Source: Electron. J. Statist., Volume 7, 244--263.

Abstract:
In linear mixed effects models, the covariance matrix of the response is modeled as the sum of two matrices: the product of the covariance matrix of the random effects with the associated design matrix, and the covariance matrix of the residual error. Building a linear mixed model usually involves selection of the parametrized covariance matrix structures for the random effects and the residual error. However, even if the covariance matrix of the response is not over-parametrized, some specifications of covariance structures can result in the non-identifiability of parameters. When fitting such models, software may or may not indicate a problem with model identifiability. Consequently, it is useful to have a way to check if a model is identifiable which does not rely on the software output. We derive conditions for identifiability of the covariance parameters of the response and study commonly used covariance structures. The derived conditions only rely on the covariance structures being used and properties of the design matrix associated with the random effects and are easy to check.

PAC-Bayesian estimation and prediction in sparse additive models

Thu, 24 Jan 2013 10:34 EST

Benjamin Guedj, Pierre Alquier

Source: Electron. J. Statist., Volume 7, 264--291.

Abstract:
The present paper is about estimation and prediction in high-dimensional additive models under a sparsity assumption ($p\gg n$ paradigm). A PAC-Bayesian strategy is investigated, delivering oracle inequalities in probability. The implementation is performed through recent outcomes in high-dimensional MCMC algorithms, and the performance of our method is assessed on simulated data.

Respondent-driven sampling on directed networks

Thu, 24 Jan 2013 10:34 EST

Xin Lu, Jens Malmros, Fredrik Liljeros, Tom Britton

Source: Electron. J. Statist., Volume 7, 292--322.

Abstract:
Respondent-driven sampling (RDS) is a widely used method for generating chain-referral samples from hidden populations. It is an extension of the snowball sampling method and can, given that some assumptions are met, generate unbiased population estimates. One key assumption, not likely to be met, is that the acquaintance network in which the recruitment process takes place is undirected, meaning that all recruiters should have the potential to be recruited by the person they recruit. Using a mean-field approach, we develop an estimator which is based on prior information about the average indegrees of estimated variables. When the indegree is known, such as for RDS studies over internet social networks, the estimator can greatly reduce estimate error and bias as compared with current methods; when the indegree is not known, which is most common for interview-based RDS studies, the estimator can through sensitivity analysis be used as a tool to account for uncertainties of network directedness and error in self-reported degree data. The performance of the new estimator, together with previous RDS estimators, is investigated thoroughly by simulations on networks with varying structures. We have applied the new estimator on an empirical RDS study for injecting drug users in New York City.

Deconvolution estimation of mixture distributions with boundaries

Mon, 28 Jan 2013 09:18 EST

Mihee Lee, Peter Hall, Haipeng Shen, J. S. Marron, Jon Tolle, Christina Burch

Source: Electron. J. Statist., Volume 7, 323--341.

Abstract:
In this paper, motivated by an important problem in evolutionary biology, we develop two sieve type estimators for distributions that are mixtures of a finite number of discrete atoms and continuous distributions under the framework of measurement error models. While there is a large literature on deconvolution problems, only two articles have previously addressed the problem taken up in our article, and they use relatively standard Fourier deconvolution. As a result the estimators suggested in those two articles are degraded seriously by boundary effects and negativity. A major contribution of our article is correct handling of boundary effects; our method is asymptotically unbiased at the boundaries, and also is guaranteed to be nonnegative. We use roughness penalization to improve the smoothness of the resulting estimator and reduce the estimation variance. We illustrate the performance of the proposed estimators via our real driving application in evolutionary biology and two simulation studies. Furthermore, we establish asymptotic properties of the proposed estimators.

Selecting the length of a principal curve within a Gaussian model

Mon, 28 Jan 2013 09:18 EST

Aurélie Fischer

Source: Electron. J. Statist., Volume 7, 342--363.

Abstract:
Principal curves are parameterized curves passing “through the middle” of a data cloud. These objects constitute a way of generalization of the notion of first principal component in Principal Component Analysis. Several definitions of principal curve have been proposed, one of which can be expressed as a least-square minimization problem. In the present paper, adopting this definition, we study a Gaussian model selection method for choosing the length of the principal curve, in order to avoid interpolation, and obtain a related oracle-type inequality. The proposed method is practically implemented and illustrated on cartography problems.

Global rates of convergence of the MLE for multivariate interval censoring

Mon, 28 Jan 2013 09:18 EST

Fuchang Gao, Jon A. Wellner

Source: Electron. J. Statist., Volume 7, 364--380.

Abstract:
We establish global rates of convergence of the Maximum Likelihood Estimator (MLE) of a multivariate distribution function on ${\mathbb{R}}^{d}$ in the case of (one type of) “interval censored” data. The main finding is that the rate of convergence of the MLE in the Hellinger metric is no worse than $n^{-1/3}(\log n)^{\gamma}$ for $\gamma =(5d-4)/6$.

Consistency of minimum description length model selection for piecewise stationary time series models

Wed, 30 Jan 2013 11:46 EST

Richard A. Davis, Chun Yip Yau

Source: Electron. J. Statist., Volume 7, 381--411.

Abstract:
This paper establishes the consistency of the minimum description length (MDL) model selection procedure by [10, 11] for a class of non-stationary time series models. We consider a time series model in which the observations are viewed as coming from stationary segments. In other words, the data are assumed to come from a general time series model in which the parameters change at break-points. Each of these segments is modeled by a pre-specified family of parametric stationary time series models. [10, 11] formulated the above problem and used the minimum description length (MDL) principle to estimate the number of break-points, the location of the break-points, the order of the parametric model and the parameter values in each of the segments. The procedure performed well on a variety of examples. In this paper we show consistency of their minimal MDL model selection procedure under general regularity conditions on the likelihood function. Results about the rate of convergence of the break-point-location estimator are also given. Applications are considered for detecting changes in independent random variables, and in ARMA and GARCH processes.

Pearson-type goodness-of-fit test with bootstrap maximum likelihood estimation

Wed, 30 Jan 2013 11:46 EST

Guosheng Yin, Yanyuan Ma

Source: Electron. J. Statist., Volume 7, 412--427.

Abstract:
The Pearson test statistic is constructed by partitioning the data into bins and computing the difference between the observed and expected counts in these bins. If the maximum likelihood estimator (MLE) of the original data is used, the statistic generally does not follow a chi-squared distribution or any explicit distribution. We propose a bootstrap-based modification of the Pearson test statistic to recover the chi-squared distribution. We compute the observed and expected counts in the partitioned bins by using the MLE obtained from a bootstrap sample. This bootstrap-sample MLE adjusts exactly the right amount of randomness to the test statistic, and recovers the chi-squared distribution. The bootstrap chi-squared test is easy to implement, as it only requires fitting exactly the same model to the bootstrap data to obtain the corresponding MLE, and then constructs the bin counts based on the original data. We examine the test size and power of the new model diagnostic procedure using simulation studies and illustrate it with a real data set.

Block thresholding for wavelet-based estimation of function derivatives from a heteroscedastic multichannel convolution model

Wed, 30 Jan 2013 11:46 EST

Fabien Navarro, Christophe Chesneau, Jalal Fadili, Taoufik Sassi

Source: Electron. J. Statist., Volume 7, 428--453.

Abstract:
We observe $n$ heteroscedastic stochastic processes $\{Y_{v}(t)\}_{v}$, where for any $v\in\{1,\ldots,n\}$ and $t\in [0,1]$, $Y_{v}(t)$ is the convolution product of an unknown function $f$ and a known blurring function $g_{v}$ corrupted by Gaussian noise. Under an ordinary smoothness assumption on $g_{1},\ldots,g_{n}$, our goal is to estimate the $d$-th derivatives (in weak sense) of $f$ from the observations. We propose an adaptive estimator based on wavelet block thresholding, namely the “BlockJS estimator". Taking the mean integrated squared error (MISE), our main theoretical result investigates the minimax rates over Besov smoothness spaces, and shows that our block estimator can achieve the optimal minimax rate, or is at least nearly-minimax in the least favorable situation. We also report a comprehensive suite of numerical simulations to support our theoretical findings. The practical performance of our block estimator compares very favorably to existing methods of the literature on a large set of test functions.

Non-asymptotic approach to varying coefficient model

Wed, 13 Feb 2013 09:14 EST

Olga Klopp, Marianna Pensky

Source: Electron. J. Statist., Volume 7, 454--479.

Abstract:
In the present paper we consider the varying coefficient model which represents a useful tool for exploring dynamic patterns in many applications. Existing methods typically provide asymptotic evaluation of precision of estimation procedures under the assumption that the number of observations tends to infinity. In practical applications, however, only a finite number of measurements are available. In the present paper we focus on a non-asymptotic approach to the problem. We propose a novel estimation procedure which is based on recent developments in matrix estimation. In particular, for our estimator, we obtain upper bounds for the mean squared and the pointwise estimation errors. The obtained oracle inequalities are non-asymptotic and hold for finite sample size.

Model selection in regression under structural constraints

Thu, 21 Feb 2013 08:59 EST

Felix Abramovich, Vadim Grinshtein

Source: Electron. J. Statist., Volume 7, 480--498.

Abstract:
The paper considers model selection in regression under the additional structural constraints on admissible models where the number of potential predictors miht be even larger than the available sample size. We develop a Bayesian formalism which is used as a natural tool for generating a wide class of model selection criteria based on penalized least squares estimation with various complexity penalties associated with a prior on a model size. The resulting criteria are adaptive to structural constraints. We establish the upper bound for the quadratic risk of the resulting MAP estimator and the corresponding lower bound for the minimax risk over a set of admissible models of a given size. We then specify the class of priors (and, therefore, the class of complexity penalties) where for the “nearly-orthogonal” design the MAP estimator is asymptotically at least nearly-minimax (up to a log-factor) simultaneously over an entire range of sparse and dense setups. Moreover, when the numbers of admissible models are “small” (e.g., ordered variable selection) or, on the opposite, for the case of complete variable selection, the proposed estimator achieves the exact minimax rates.

Data-driven density derivative estimation, with applications to nonparametric clustering and bump hunting

Wed, 06 Mar 2013 09:16 EST

José E. Chacón, Tarn Duong

Source: Electron. J. Statist., Volume 7, 499--532.

Abstract:
Important information concerning a multivariate data set, such as clusters and modal regions, is contained in the derivatives of the probability density function. Despite this importance, nonparametric estimation of higher order derivatives of the density functions have received only relatively scant attention. Kernel estimators of density functions are widely used as they exhibit excellent theoretical and practical properties, though their generalization to density derivatives has progressed more slowly due to the mathematical intractabilities encountered in the crucial problem of bandwidth (or smoothing parameter) selection. This paper presents the first fully automatic, data-based bandwidth selectors for multivariate kernel density derivative estimators. This is achieved by synthesizing recent advances in matrix analytic theory which allow mathematically and computationally tractable representations of higher order derivatives of multivariate vector valued functions. The theoretical asymptotic properties as well as the finite sample behaviour of the proposed selectors are studied. In addition, we explore in detail the applications of the new data-driven methods for two other statistical problems: clustering and bump hunting. The introduced techniques are combined with the mean shift algorithm to develop novel automatic, nonparametric clustering procedures which are shown to outperform mixture-model cluster analysis and other recent nonparametric approaches in practice. Furthermore, the advantage of the use of smoothing parameters designed for density derivative estimation for feature significance analysis for bump hunting is illustrated with a real data example.

Noise recovery for Lévy-driven CARMA processes and high-frequency behaviour of approximating Riemann sums

Wed, 06 Mar 2013 09:16 EST

Vincenzo Ferrazzano, Florian Fuchs

Source: Electron. J. Statist., Volume 7, 533--561.

Abstract:
We consider high-frequency sampled continuous-time autoregressive moving average (CARMA) models driven by finite-variance zero-mean Lévy processes. An $L^{2}$-consistent estimator for the increments of the driving Lévy process without order selection in advance is proposed if the CARMA model is invertible. In the second part we analyse the high-frequency behaviour of approximating Riemann sum processes, which represent a natural way to simulate continuous-time moving average models on a discrete grid. We compare their autocovariance structure with the one of sampled CARMA processes and show that the rule of integration plays a crucial role. Moreover, new insight into the kernel estimation procedure of Brockwell et al. [11] is given.

Uniform convergence and asymptotic confidence bands for model-assisted estimators of the mean of sampled functional data

Thu, 14 Mar 2013 09:42 EDT

Hervé Cardot, Camelia Goga, Pauline Lardin

Source: Electron. J. Statist., Volume 7, 562--596.

Abstract:
When the study variable is functional and storage capacities are limited or transmission costs are high, selecting with survey sampling techniques a small fraction of the observations is an interesting alternative to signal compression techniques, particularly when the goal is the estimation of simple quantities such as means or totals. We extend, in this functional framework, model-assisted estimators with linear regression models that can take account of auxiliary variables whose totals over the population are known. We first show, under weak hypotheses on the sampling design and the regularity of the trajectories, that the estimator of the mean function as well as its variance estimator are uniformly consistent. Then, under additional assumptions, we prove a functional central limit theorem and we assess rigorously a fast technique based on simulations of Gaussian processes which is employed to build asymptotic confidence bands. The accuracy of the variance function estimator is evaluated on a real dataset of sampled electricity consumption curves measured every half an hour over a period of one week.

Needlet-Whittle estimates on the unit sphere

Thu, 14 Mar 2013 09:42 EDT

Claudio Durastanti, Xiaohong Lan, Domenico Marinucci

Source: Electron. J. Statist., Volume 7, 597--646.

Abstract:
We study the asymptotic behaviour of needlets-based approximate maximum likelihood estimators for the spectral parameters of Gaussian and isotropic spherical random fields. We prove consistency and asymptotic Gaussianity, in the high-frequency limit, thus generalizing earlier results by Durastanti et al. (2011) based upon standard Fourier analysis on the sphere. The asymptotic results are then illustrated by an extensive Monte Carlo study.

On parameter estimation for critical affine processes

Thu, 14 Mar 2013 09:42 EDT

Mátyás Barczy, Leif Döring, Zenghu Li, Gyula Pap

Source: Electron. J. Statist., Volume 7, 647--696.

Abstract:
First we provide a simple set of sufficient conditions for the weak convergence of scaled affine processes with state space $\mathbb{R}_{+}\times \mathbb{R}^{d}$. We specialize our result to one-dimensional continuous state branching processes with immigration. As an application, we study the asymptotic behavior of least squares estimators of some parameters of a two-dimensional critical affine diffusion process.

Marginals of multivariate Gibbs distributions with applications in Bayesian species sampling

Sat, 16 Mar 2013 21:07 EDT

Annalisa Cerquetti

Source: Electron. J. Statist., Volume 7, 697--716.

Abstract:
Gibbs partition models are the largest class of infinite exchangeable partitions of the positive integers generalizing the product form of the probability function of the two-parameter Poisson-Dirichlet family. Here we call into question the current approach to Bayesian nonparametric estimation in species sampling problems under Gibbs priors , which incorrectly relies on treating exchangeable partition probability functions (EPPFs) as multivariate distributions on compositions of the positive integers. We show that once those multivariate distributions are correctly derived, results for corresponding sampling formulas can be obtained, generalized and sometimes fixed, working with marginals and a known result on falling factorial moments of a sum of non independent indicators. We provide an application of our findings to a recently proposed Bayesian nonparametric estimation under Gibbs priors of the predictive probability to observe a species already observed a certain number of times.

Bayes minimax estimation under power priors of location parameters for a wide class of spherically symmetric distributions

Mon, 25 Mar 2013 10:12 EDT

Dominique Fourdrinier, Fatiha Mezoued, William E. Strawderman

Source: Electron. J. Statist., Volume 7, 717--741.

Abstract:
We complement the results of Fourdrinier, Mezoued and Strawderman in [5] who considered Bayesian estimation of the location parameter $\theta$ of a random vector $X$ having a unimodal spherically symmetric density $f(\|x-\theta\|^{2})$ for a spherically symmetric prior density $\pi(\|\theta\|^{2})$. In [5], expressing the Bayes estimator as $\delta_{\pi}(X)=X+\nabla M(\|X\|^{2})/m(\|X\|^{2})$, where $m$ is the marginal associated to $f(\|x-\theta\|^{2})$ and $M$ is the marginal with respect to $F(\|x-\theta\|^{2})=1/2\int_{\|x-\theta\|^{2}}^{\infty}f(t)\,dt$, it was shown that, under quadratic loss, if the sampling density $f(\|x-\theta\|^{2})$ belongs to the Berger class (i.e. there exists a positive constant $c$ such that $F(t)/f(t)\geq c$ for all $t$), conditions, dependent on the monotonicity of the ratio $F(t)/f(t)$, can be found on $\pi$ in order that $\delta_{\pi}(X)$ is minimax. The main feature of this paper is that, in the case where $F(t)/f(t)$ is nonincreasing, if $\pi(\|\theta\|^{2})$ is a superharmonic power prior of the form $\|\theta\|^{-2k}$ with $k>0$, the membership of the sampling density to the Berger class can be droped out. Also, our techniques are different from those in [5]. First, writing $\delta_{\pi}(X)=X+g(X)$ with $g(X)\propto \nabla M(\|X\|^{2})/m(\|X\|^{2})$, we follow Brandwein and Strawderman [4] proving that, for some $b>0$, the function $h=b\,\Delta M/m$ is subharmonic and satisfies $\|g\|^{2}/2\leq -h\leq -{\rm div}g$. Also, we adapt their approach using the fact that $R^{2(k+1)}\int_{B_{\theta,R}}h(x)\,dV_{\theta,R}(x)$ is nonincreasing in $R$ for any $\theta \in{\mathbb{R}} ^{p}$, when $V_{\theta,R}$ is the uniform distribution on the ball $B_{\theta,R}$ of radius $R$ and centered at $\theta$. Examples illustrate the theory.

Generalized predictive information criteria for the analysis of feature events

Mon, 25 Mar 2013 10:12 EDT

Mike K. P. So, Tomohiro Ando

Source: Electron. J. Statist., Volume 7, 742--762.

Abstract:
This paper develops two weighted measures for model selection by generalizing the Kullback-Leibler divergence measure. The concept of a model selection process that takes into account the special features of the underlying model is introduced using weighted measures. New information criteria are defined using the bias correction of an expected weighted loglikelihood estimator. Using weight functions that match the features of interest in the underlying statistical models, the new information criteria are applied to simulated studies of spline regression and copula model selection. Real data applications are also given for predicting the incidence of disease and for quantile modeling of environmental data.

Online Expectation Maximization based algorithms for inference in Hidden Markov Models

Mon, 25 Mar 2013 10:12 EDT

Sylvain Le Corff, Gersende Fort

Source: Electron. J. Statist., Volume 7, 763--792.

Abstract:
The Expectation Maximization (EM) algorithm is a versatile tool for model parameter estimation in latent data models. When processing large data sets or data stream however, EM becomes intractable since it requires the whole data set to be available at each iteration of the algorithm. In this contribution, a new generic online EM algorithm for model parameter inference in general Hidden Markov Model is proposed. This new algorithm updates the parameter estimate after a block of observations is processed (online). The convergence of this new algorithm is established, and the rate of convergence is studied showing the impact of the block-size sequence. An averaging procedure is also proposed to improve the rate of convergence. Finally, practical illustrations are presented to highlight the performance of these algorithms in comparison to other online maximum likelihood procedures.

A goodness-of-fit test for Poisson count processes

Mon, 25 Mar 2013 10:12 EDT

Konstantinos Fokianos, Michael H. Neumann

Source: Electron. J. Statist., Volume 7, 793--819.

Abstract:
We are studying a novel class of goodness-of-fit tests for parametric count time series regression models. These test statistics are formed by considering smoothed versions of the empirical process of the Pearson residuals. Our construction yields test statistics which are consistent against Pitman’s local alternatives and they converge weakly at the usual parametric rate. To approximate the asymptotic null distribution of the test statistics, we propose a parametric bootstrap method and we study its properties. The methodology is applied to simulated and real data.

A recursive procedure for density estimation on the binary hypercube

Mon, 25 Mar 2013 10:12 EDT

Maxim Raginsky, Jorge G. Silva, Svetlana Lazebnik, Rebecca Willett

Source: Electron. J. Statist., Volume 7, 820--858.

Abstract:
This paper describes a recursive estimation procedure for multivariate binary densities (probability distributions of vectors of Bernoulli random variables) using orthogonal expansions. For $d$ covariates, there are $2^{d}$ basis coefficients to estimate, which renders conventional approaches computationally prohibitive when $d$ is large. However, for a wide class of densities that satisfy a certain sparsity condition, our estimator runs in probabilistic polynomial time and adapts to the unknown sparsity of the underlying density in two key ways: (1) it attains near-minimax mean-squared error for moderate sample sizes, and (2) the computational complexity is lower for sparser densities. Our method also allows for flexible control of the trade-off between mean-squared error and computational complexity.

Two-component mixtures with independent coordinates as conditional mixtures: Nonparametric identification and estimation

Wed, 03 Apr 2013 09:05 EDT

Daniel Hohmann, Hajo Holzmann

Source: Electron. J. Statist., Volume 7, 859--880.

Abstract:
We show how the multivariate two-component mixtures with independent coordinates in each component by Hall and Zhou (2003) can be studied within the framework of conditional mixtures as recently introduced by Henry, Kitamura and Salanié (2010). Here, the conditional distribution of the random variable $Y$ given the vector of regressors $Z$ can be expressed as a two-component mixture, where only the mixture weights depend on the covariates. Under appropriate tail conditions on the characteristic functions and the distribution functions of the mixture components, which allow for flexible location-scale type mixtures, we show identification and provide asymptotically normal estimators. The main application for our results are bivariate two-component mixtures with independent coordinates, the case not previously covered by Hall and Zhou (2003). In a simulation study we investigate the finite-sample performance of the proposed methods. The main new technical ingredient is the estimation of limits of quotients of two characteristic functions in the tails from independent samples, which might be of some independent interest.

Intensity estimation of non-homogeneous Poisson processes from shifted trajectories

Wed, 03 Apr 2013 09:05 EDT

Jérémie Bigot, Sébastien Gadat, Thierry Klein, Clément Marteau

Source: Electron. J. Statist., Volume 7, 881--931.

Abstract:
In this paper, we consider the problem of estimating nonparametrically a mean pattern intensity $\lambda$ from the observation of $n$ independent and non-homogeneous Poisson processes $N^{1},\dots,N^{n}$ on the interval $[0,1]$. This problem arises when data (counts) are collected independently from $n$ individuals according to similar Poisson processes. We show that estimating this intensity is a deconvolution problem for which the density of the random shifts plays the role of the convolution operator. In an asymptotic setting where the number $n$ of observed trajectories tends to infinity, we derive upper and lower bounds for the minimax quadratic risk over Besov balls. Non-linear thresholding in a Meyer wavelet basis is used to derive an adaptive estimator of the intensity. The proposed estimator is shown to achieve a near-minimax rate of convergence. This rate depends both on the smoothness of the intensity function and the density of the random shifts, which makes a connection between the classical deconvolution problem in nonparametric statistics and the estimation of a mean intensity from the observations of independent Poisson processes.

Weighted least squares estimation with missing responses: An empirical likelihood approach

Wed, 03 Apr 2013 09:05 EDT

Anton Schick

Source: Electron. J. Statist., Volume 7, 932--945.

Abstract:
A heteroscedastic linear regression model is considered where responses are allowed to be missing at random. An estimator is constructed that matches the performance of the weighted least squares estimator without the knowledge of the conditional variance function. This is usually done by constructing an estimator of the variance function. Our estimator is a maximum empirical likelihood estimator based on an increasing number of estimated constraints and avoids estimating the variance function.

Spatial models for point and areal data using Markov random fields on a fine grid

Mon, 15 Apr 2013 09:05 EDT

Christopher J. Paciorek

Source: Electron. J. Statist., Volume 7, 946--972.

Abstract:
I consider the use of Markov random fields (MRFs) on a fine grid to represent latent spatial processes when modeling point-level and areal data, including situations with spatial misalignment. Point observations are related to the grid cell in which they reside, while areal observations are related to the (approximate) integral over the latent process within the area of interest. I review several approaches to specifying the neighborhood structure for constructing the MRF precision matrix, presenting results comparing these MRF representations analytically, in simulations, and in two examples. The results provide practical guidance for choosing a spatial process representation and highlight the importance of this choice. In particular, the results demonstrate that, and explain why, standard CAR models can behave strangely for point-level data. They show that various neighborhood weighting approaches based on higher-order neighbors that have been suggested for MRF models do not produce smooth fields, which raises doubts about their utility. Finally, they indicate that an MRF that approximates a thin plate spline compares favorably to standard CAR models and to kriging under many circumstances.

Hierarchical Bayes, maximum a posteriori estimators, and minimax concave penalized likelihood estimation

Mon, 15 Apr 2013 09:05 EDT

Robert L. Strawderman, Martin T. Wells, Elizabeth D. Schifano

Source: Electron. J. Statist., Volume 7, 973--990.

Abstract:
Priors constructed from scale mixtures of normal distributions have long played an important role in decision theory and shrinkage estimation. This paper demonstrates equivalence between the maximum aposteriori estimator constructed under one such prior and Zhang’s minimax concave penalization estimator. This equivalence and related multivariate generalizations stem directly from an intriguing representation of the minimax concave penalty function as the Moreau envelope of a simple convex function. Maximum aposteriori estimation under the corresponding marginal prior distribution, a generalization of the quasi-Cauchy distribution proposed by Johnstone and Silverman, leads to thresholding estimators having excellent frequentist risk properties.

Empirical Bayes scaling of Gaussian priors in the white noise model

Mon, 15 Apr 2013 09:05 EDT

B. T. Szabó, A. W. van der Vaart, J. H. van Zanten

Source: Electron. J. Statist., Volume 7, 991--1018.

Abstract:
The performance of nonparametric estimators is heavily dependent on a bandwidth parameter. In nonparametric Bayesian methods this parameter can be specified as a hyperparameter of the nonparametric prior. The value of this hyperparameter may be made dependent on the data. The empirical Bayes method is to set its value by maximizing the marginal likelihood of the data in the Bayesian framework. In this paper we analyze a particular version of this method, common in practice, that the hyperparameter scales the prior variance. We characterize the behavior of the random hyperparameter, and show that a nonparametric Bayes method using it gives optimal recovery over a scale of regularity classes. This scale is limited, however, by the regularity of the unscaled prior. While a prior can be scaled up to make it appropriate for arbitrarily rough truths, scaling cannot increase the nominal smoothness by much. Surprisingy the standard empirical Bayes method is even more limited in this respect than an oracle, deterministic scaling method. The same can be said for the hierarchical Bayes method.

Time series clustering based on nonparametric multidimensional forecast densities

Mon, 15 Apr 2013 09:05 EDT

José A. Vilar, Juan M. Vilar

Source: Electron. J. Statist., Volume 7, 1019--1046.

Abstract:
A new time series clustering method based on comparing forecast densities for a sequence of $k>1$ consecutive horizons is proposed. The unknown $k$-dimensional forecast densities can be non-parametrically approximated by using bootstrap procedures that mimic the generating processes without parametric restrictions. However, the difficulty of constructing accurate kernel estimators of multivariate densities is well known. To circumvent the high dimensionality problem, the bootstrap prediction vectors are projected onto a lower-dimensional space using principal components analysis, and then the densities are estimated in this new space. Proper distances between pairs of estimated densities are computed and used to generate an initial dissimilarity matrix, and hence a standard hierarchical clustering is performed. The clustering procedure is examined via simulation and is applied to a real dataset involving electricity prices series.

Consistent model selection of discrete Bayesian networks from incomplete data

Mon, 15 Apr 2013 09:05 EDT

Nikolay Balov

Source: Electron. J. Statist., Volume 7, 1047--1077.

Abstract:
A maximum likelihood based model selection of discrete Bayesian networks is considered. The structure learning is performed by employing a scoring function $S$, which, for a given network $G$ and $n$-sample $D_{n}$, is defined as the maximum marginal log-likelihood $l$ minus a penalization term $\lambda_{n}h$ proportional to network complexity $h(G)$, $$S(G|D_{n})=l(G|D_{n})-\lambda_{n}h(G).$$ An available case analysis is developed with the standard log-likelihood replaced by the sum of sample average node log-likelihoods. The approach utilizes partially missing data records and allows for comparison of models fitted to different samples. In missing completely at random settings the estimation is shown to be consistent if and only if the sequence $\lambda_{n}$ converges to zero at a slower than $n^{-{1/2}}$ rate. In particular, the BIC model selection ($\lambda_{n}=0.5\log(n)/n$) applied to the node-average log-likelihood is shown to be inconsistent in general. This is in contrast to the complete data case when BIC is known to be consistent. The conclusions are confirmed by numerical experiments.

Asymptotics of a clustering criterion for smooth distributions

Mon, 15 Apr 2013 09:05 EDT

Karthik Bharath, Vladimir Pozdnyakov, Dipak K. Dey

Source: Electron. J. Statist., Volume 7, 1078--1093.

Abstract:
We develop a clustering framework for observations from a population with a smooth probability distribution function and derive its asymptotic properties. A clustering criterion based on a linear combination of order statistics is proposed. The asymptotic behavior of the point at which the observations are split into two clusters is examined. The results obtained can then be utilized to construct an interval estimate of the point which splits the data and develop tests for bimodality and presence of clusters.

Laplace deconvolution with noisy observations

Mon, 22 Apr 2013 09:58 EDT

Felix Abramovich, Marianna Pensky, Yves Rozenholc

Source: Electron. J. Statist., Volume 7, 1094--1128.

Abstract:
In the present paper we consider Laplace deconvolution problem for discrete noisy data observed on an interval whose length $T_{n}$ may increase with the sample size. Although this problem arises in a variety of applications, to the best of our knowledge, it has been given very little attention by the statistical community. Our objective is to fill the gap and provide statistical analysis of Laplace deconvolution problem with noisy discrete data. The main contribution of the paper is an explicit construction of an asymptotically rate-optimal (in the minimax sense) Laplace deconvolution estimator which is adaptive to the regularity of the unknown function. We show that the original Laplace deconvolution problem can be reduced to nonparametric estimation of a regression function and its derivatives on the interval of growing length $T_{n}$. Whereas the forms of the estimators remain standard, the choices of the parameters and the minimax convergence rates, which are expressed in terms of $T_{n}^{2}/n$ in this case, are affected by the asymptotic growth of the length of the interval. We derive an adaptive kernel estimator of the function of interest, and establish its asymptotic minimaxity over a range of Sobolev classes. We illustrate the theory by examples of construction of explicit expressions of Laplace deconvolution estimators. A simulation study shows that, in addition to providing asymptotic optimality as the number of observations tends to infinity, the proposed estimator demonstrates good performance in finite sample examples.

An estimation of the stability and the localisability functions of multistable processes

Mon, 22 Apr 2013 09:58 EDT

R. Le Guével

Source: Electron. J. Statist., Volume 7, 1129--1166.

Abstract:
Multistable processes are tangent at each point to a stable process, but where the index of stability and the index of localisability varies along the path. In this work, we give two estimators of the stability and the localisability functions, and we prove the consistency of those two estimators. We illustrate these convergences with two examples, the Lévy multistable process and the Linear Multifractional Multistable Motion.

A wavelet-based approach for detecting changes in second order structure within nonstationary time series

Tue, 23 Apr 2013 09:03 EDT

R. Killick, I. A. Eckley, P. Jonathan

Source: Electron. J. Statist., Volume 7, 1167--1183.

Abstract:
This article proposes a test to detect changes in general autocovariance structure in nonstationary time series. Our approach is founded on the locally stationary wavelet (LSW) process model for time series which has previously been used for classification and segmentation of time series. Using this framework we form a likelihood-based hypothesis test and demonstrate its performance against existing methods on various simulated examples as well as applying it to a problem arising from ocean engineering.

Optimal model selection in heteroscedastic regression using piecewise polynomial functions

Thu, 25 Apr 2013 09:35 EDT

Adrien Saumard

Source: Electron. J. Statist., Volume 7, 1184--1223.

Abstract:
We consider the estimation of a regression function with random design and heteroscedastic noise in a nonparametric setting. More precisely, we address the problem of characterizing the optimal penalty when the regression function is estimated by using a penalized least-squares model selection method. In this context, we show the existence of a minimal penalty, defined to be the maximum level of penalization under which the model selection procedure totally misbehaves. The optimal penalty is shown to be twice the minimal one and to satisfy a non-asymptotic pathwise oracle inequality with leading constant almost one. Finally, the ideal penalty being unknown in general, we propose a hold-out penalization procedure and show that the latter is asymptotically optimal.

Regenerative block-bootstrap confidence intervals for tail and extremal indexes

Thu, 25 Apr 2013 09:35 EDT

Patrice Bertail, Stéphan Clémençon, Jessica Tressou

Source: Electron. J. Statist., Volume 7, 1224--1248.

Abstract:
A theoretically sound bootstrap procedure is proposed for building accurate confidence intervals of parameters describing the extremal behavior of instantaneous functionals $\{f(X_{n})\}_{n\in\mathbb{N}}$ of a Harris Markov chain $X$, namely the extremal and tail indexes. Regenerative properties of the chain $X$ (or of a Nummelin extension of the latter) are here exploited in order to construct consistent estimators of these parameters, following the approach developed in [10]. Their asymptotic normality is first established and the standardization problem is also tackled. It is then proved that, based on these estimators, the regenerative block-bootstrap and its approximate version, both introduced in [7], yield asymptotically valid confidence intervals. In order to illustrate the performance of the methodology studied in this paper, simulation results are additionally displayed.

Upper bounds and aggregation in bipartite ranking

Mon, 29 Apr 2013 09:30 EDT

Sylvain Robbiano

Source: Electron. J. Statist., Volume 7, 1249--1271.

Abstract:
One main focus of learning theory is to find optimal rates of convergence. In classification, it is possible to obtain optimal fast rates (faster than $n^{-1/2}$) in a minimax sense. Moreover, using an aggregation procedure, the algorithms are adaptive to the parameters of the class of distributions. Here, we investigate this issue in the bipartite ranking framework. We design a ranking rule by aggregating estimators of the regression function. We use exponential weights based on the empirical ranking risk. Under several assumptions on the class of distribution, we show that this procedure is adaptive to the margin parameter and smoothness parameter and achieves the same rates as in the classification framework. Moreover, we state a minimax lower bound that establishes the optimality of the aggregation procedure in a specific case.

Bayes multiple decision functions

Fri, 03 May 2013 08:44 EDT

Wensong Wu, Edsel A. Peña

Source: Electron. J. Statist., Volume 7, 1272--1300.

Abstract:
This paper deals with the problem of simultaneously making many ($M$) binary decisions based on one realization of a random data matrix $\mathbf{X}$. $M$ is typically large and $\mathbf{X}$ will usually have $M$ rows associated with each of the $M$ decisions to make, but for each row the data may be low dimensional. Such problems arise in many practical areas such as the biological and medical sciences, where the available dataset is from microarrays or other high-throughput technology and with the goal being to decide which among of many genes are relevant with respect to some phenotype of interest; in the engineering and reliability sciences; in astronomy; in education; and in business. A Bayesian decision-theoretic approach to this problem is implemented with the overall loss function being a cost-weighted linear combination of Type I and Type II loss functions. The class of loss functions considered allows for use of the false discovery rate (FDR), false nondiscovery rate (FNR), and missed discovery rate (MDR) in assessing the quality of decision. Through this Bayesian paradigm, the Bayes multiple decision function (BMDF) is derived and an efficient algorithm to obtain the optimal Bayes action is described. In contrast to many works in the literature where the rows of the matrix $\mathbf{X}$ are assumed to be stochastically independent, we allow a dependent data structure with the associations obtained through a class of frailty-induced Archimedean copulas. In particular, non-Gaussian dependent data structure, which is typical with failure-time data, can be entertained. The numerical implementation of the determination of the Bayes optimal action is facilitated through sequential Monte Carlo techniques. The theory developed could also be extended to the problem of multiple hypotheses testing, multiple classification and prediction, and high-dimensional variable selection. The proposed procedure is illustrated for the simple versus simple hypotheses setting and for the composite hypotheses setting through simulation studies. The procedure is also applied to a subset of a microarray data set from a colon cancer study.

Adaptive estimation of convex polytopes and convex sets from noisy data

Fri, 10 May 2013 09:46 EDT

Victor-Emmanuel Brunel

Source: Electron. J. Statist., Volume 7, 1301--1327.

Abstract:
We estimate convex polytopes and general convex sets in $\mathbb{R}^{d}$, $d\geq 2$ in the regression framework. We measure the risk of our estimators using a $L^{1}$-type loss function and prove upper bounds on these risks. We show, in the case of convex polytopes, that these estimators achieve the minimax rate. For convex polytopes, this minimax rate is $\frac{\ln n}{n}$, which differs from the parametric rate for non-regular families by a logarithmic factor, and we show that this extra factor is essential. Using polytopal approximations we extend our results to general convex sets, and we achieve the minimax rate up to a logarithmic factor. In addition we provide an estimator that is adaptive with respect to the number of vertices of the unknown polytope, and we prove that this estimator is optimal in all classes of convex polytopes with a given number of vertices.

Some optimality properties of FDR controlling rules under sparsity

Fri, 10 May 2013 09:46 EDT

Florian Frommlet, Małgorzata Bogdan

Source: Electron. J. Statist., Volume 7, 1328--1368.

Abstract:
False Discovery Rate (FDR) and the Bayes risk are two different statistical measures, which can be used to evaluate and compare multiple testing procedures. Recent results show that under sparsity FDR controlling procedures, like the popular Benjamini-Hochberg (BH) procedure, perform also very well in terms of the Bayes risk. In particular asymptotic Bayes optimality under sparsity (ABOS) of BH was shown previously for location and scale models based on log-concave densities. This article extends previous work to a substantially larger set of distributions of effect sizes under the alternative, where the alternative distribution of true signals does not change with the number of tests $m$, while the sample size $n$ slowly increases. ABOS of BH and the corresponding step-down procedure based on FDR levels proportional to $n^{-1/2}$ are proved. A simulation study shows that these asymptotic results are relevant already for relatively small values of $m$ and $n$. Apart from showing asymptotic optimality of BH, our results on the optimal FDR level provide a natural extension of the well known results on the significance levels of Bayesian tests.

PPtree: Projection pursuit classification tree

Fri, 10 May 2013 09:46 EDT

Yoon Dong Lee, Dianne Cook, Ji-won Park, Eun-Kyung Lee

Source: Electron. J. Statist., Volume 7, 1369--1386.

Abstract:
In this paper, we propose a new classification tree, the projection pursuit classification tree (PPtree). It combines tree structured methods with projection pursuit dimension reduction. This tree is originated from the projection pursuit method for classification. In each node, one of the projection pursuit indices using class information - LDA, $L_{r}$ or PDA indices - is maximized to find the projection with the most separated group view. On this optimized data projection, the tree splitting criteria are applied to separate the groups. These steps are iterated until the last two classes are separated. The main advantages of this tree is that it effectively uses correlation between variables to find separations, and it has visual representation of the differences between groups in a 1-dimensional space that can be used to interpret results. Also in each node of the tree, the projection coefficients represent the variable importance for the group separation. This information is very helpful to select variables in classification problems.

Nadaraya-Watson estimator for stochastic processes driven by stable Lévy motions

Fri, 10 May 2013 09:46 EDT

Hongwei Long, Lianfen Qian

Source: Electron. J. Statist., Volume 7, 1387--1418.

Abstract:
We discuss the nonparametric Nadaraya-Watson (N-W) estimator of the drift function for ergodic stochastic processes driven by $\alpha$-stable noises and observed at discrete instants. Under geometrical mixing condition, we derive consistency and rate of convergence of the N-W estimator of the drift function. Furthermore, we obtain a central limit theorem for stable stochastic integrals. The central limit theorem has its own interest and is the crucial tool for the proofs. A simulation study illustrates the finite sample properties of the N-W estimator.

Optimal regression rates for SVMs using Gaussian kernels

Tue, 14 May 2013 09:50 EDT

Mona Eberts, Ingo Steinwart

Source: Electron. J. Statist., Volume 7, 1--42.

Abstract:
Support vector machines (SVMs) using Gaussian kernels are one of the standard and state-of-the-art learning algorithms. In this work, we establish new oracle inequalities for such SVMs when applied to either least squares or conditional quantile regression. With the help of these oracle inequalities we then derive learning rates that are (essentially) minmax optimal under standard smoothness assumptions on the target function. We further utilize the oracle inequalities to show that these learning rates can be adaptively achieved by a simple data-dependent parameter selection method that splits the data set into a training and a validation set.

Gradient statistic: Higher-order asymptotics and Bartlett-type correction

Tue, 14 May 2013 09:50 EDT

Tiago M. Vargas, Silvia L.P. Ferrari, Artur J. Lemonte

Source: Electron. J. Statist., Volume 7, 43--61.

Abstract:
We obtain an asymptotic expansion for the null distribution function of the gradient statistic for testing composite null hypotheses in the presence of nuisance parameters. The expansion is derived using a Bayesian route based on the shrinkage argument described in [10]. Using this expansion, we propose a Bartlett-type corrected gradient statistic with chi-square distribution up to an error of order $o(n^{-1})$ under the null hypothesis. Further, we also use the expansion to modify the percentage points of the large sample reference chi-square distribution. Monte Carlo simulation experiments and various examples are presented and discussed.

A vector of Dirichlet processes

Tue, 21 May 2013 11:04 EDT

Fabrizio Leisen, Antonio Lijoi, Dario Spanó

Source: Electron. J. Statist., Volume 7, 62--90.

Abstract:
Random probability vectors are of great interest especially in view of their application to statistical inference. Indeed, they can be used for identifying the de Finetti mixing measure in the representation of the law of a partially exchangeable array of random elements taking values in a separable and complete metric space. In this paper we describe the construction of a vector of Dirichlet processes based on the normalization of an exchangeable vector of completely random measures that are jointly infinitely divisible. After deducing the form of the multivariate Laplace exponent associated to the vector of the gamma completely random measures, we analyze some of their distributional properties. Our attention particularly focuses on the dependence structure and the specific partition probability function induced by the proposed vector.

Cartesian displays of many interval estimates

Tue, 21 May 2013 11:04 EDT

Mario Peruggia, Jason Hsu, Yifan Huang

Source: Electron. J. Statist., Volume 7, 91--104.

Abstract:
We consider the problem of constructing static graphical representations of a large number of interval estimates. Because of clutter, traditional graphical summaries are visually ineffective for representing more then a few intervals. The Cartesian displays introduced in this article overcome the limitations stemming from visual clutter and can represent effectively very many intervals. The construction of a Cartesian display for symmetric intervals is first presented in the context of a multiple comparisons application. Generalizations involving the representation of asymmetric intervals are then introduced and used to summarize aspects of the posterior distributions of numerous parameter contrasts in two hierarchical Bayes models.

On Bayesian credible sets, restricted parameter spaces and frequentist coverage

Tue, 21 May 2013 11:04 EDT

Éric Marchand, William E. Strawderman

Source: Electron. J. Statist., Volume 7, 1419--1431.

Abstract:
For estimating a lower bounded parametric function in the framework of Marchand and Strawderman [6], we provide through a unified approach a class of Bayesian confidence intervals with credibility $1-\alpha$ and frequentist coverage probability bounded below by $\frac{1-\alpha}{1+\alpha}$. In cases where the underlying pivotal distribution is symmetric, the findings represent extensions with respect to the specification of the credible set achieved through the choice of a spending function , and include Marchand and Strawderman’s HPD procedure result. For non-symmetric cases, the determination of a such a class of Bayesian credible sets fills a gap in the literature and includes an “equal-tails” modification of the HPD procedure. Several examples are presented demonstrating wide applicability.

A penalized likelihood estimation approach to semiparametric sample selection binary response modeling

Tue, 21 May 2013 11:04 EDT

Giampiero Marra, Rosalba Radice

Source: Electron. J. Statist., Volume 7, 1432--1455.

Abstract:
Sample selection models are employed when an outcome of interest is observed for a restricted non-randomly selected sample of the population. We consider the case in which the response is binary and continuous covariates have a nonlinear relationship to the outcome. We introduce two statistical methods for the estimation of two binary regression models involving semiparametric predictors in the presence of non-random sample selection. This is achieved using a multiple-stage procedure, and a newly developed simultaneous equation estimation scheme. Both approaches are based on the penalized likelihood estimation framework. The problems of identification and inference are also discussed. The empirical properties of the proposed approaches are studied through a simulation study. The methods are then illustrated using data from the American National Election Study where the aim is to quantify public support for school integration. If non-random sample selection is neglected then the predicted probability of giving, for instance, a supportive response may be biased, an issue that can be tackled using the proposed tools.

The lasso problem and uniqueness

Tue, 21 May 2013 11:04 EDT

Ryan J. Tibshirani

Source: Electron. J. Statist., Volume 7, 1456--1490.

Abstract:
The lasso is a popular tool for sparse linear regression, especially for problems in which the number of variables $p$ exceeds the number of observations $n$. But when $p>n$, the lasso criterion is not strictly convex, and hence it may not have a unique minimizer. An important question is: when is the lasso solution well-defined (unique)? We review results from the literature, which show that if the predictor variables are drawn from a continuous probability distribution, then there is a unique lasso solution with probability one, regardless of the sizes of $n$ and $p$. We also show that this result extends easily to $\ell_{1}$ penalized minimization problems over a wide range of loss functions. A second important question is: how can we manage the case of non-uniqueness in lasso solutions? In light of the aforementioned result, this case really only arises when some of the predictor variables are discrete, or when some post-processing has been performed on continuous predictor measurements. Though we certainly cannot claim to provide a complete answer to such a broad question, we do present progress towards understanding some aspects of non-uniqueness. First, we extend the LARS algorithm for computing the lasso solution path to cover the non-unique case, so that this path algorithm works for any predictor matrix. Next, we derive a simple method for computing the component-wise uncertainty in lasso solutions of any given problem instance, based on linear programming. Finally, we review results from the literature on some of the unifying properties of lasso solutions, and also point out particular forms of solutions that have distinctive properties.

Presmoothing the Aalen-Johansen estimator in the illness-death model

Tue, 21 May 2013 11:04 EDT

Ana Moreira, Jacobo de Uña-Álvarez, Luís Machado

Source: Electron. J. Statist., Volume 7, 1491--1516.

Abstract:
One major goal in clinical applications of multi-state models is the estimation of transition probabilities. The usual nonparametric estimator of the transition matrix for non-homogeneous Markov processes is the Aalen-Johansen estimator (Aalen and Johansen 1978 [1]). In this paper we propose a modification of the Aalen-Johansen estimator in the illness-death model based on presmoothing. The consistency of the proposed estimators is formally established. Simulations show that the presmoothed estimators may be much more efficient than the Aalen-Johansen estimator. A real data illustration is included.