Open Access
2022 Aggregated hold out for sparse linear regression with a robust loss function
Guillaume Maillard
Author Affiliations +
Electron. J. Statist. 16(1): 935-997 (2022). DOI: 10.1214/21-EJS1952


Sparse linear regression methods generally have a free hyperparameter which controls the amount of sparsity, and is subject to a bias-variance tradeoff. This article considers the use of Aggregated hold-out to aggregate over values of this hyperparameter, in the context of linear regression with the Huber loss function. Aggregated hold-out (Agghoo) is a procedure which averages estimators selected by hold-out (cross-validation with a single split). In the theoretical part of the article, it is proved that Agghoo satisfies a non-asymptotic oracle inequality when it is applied to sparse estimators which are parametrized by their zero-norm. In particular, this includes a variant of the Lasso introduced by Zou, Hastié and Tibshirani [49]. Simulations are used to compare Agghoo with cross-validation. They show that Agghoo performs better than CV when the intrinsic dimension is high and when there are confounders correlated with the predictive covariates.

Funding Statement

While finishing the writing of this article, the author (Guillaume Maillard) has received funding from the European Union’s Horizon 2020 research and innovation program under grant agreement No 811017.


Download Citation

Guillaume Maillard. "Aggregated hold out for sparse linear regression with a robust loss function." Electron. J. Statist. 16 (1) 935 - 997, 2022.


Received: 1 February 2020; Published: 2022
First available in Project Euclid: 3 February 2022

MathSciNet: MR4377131
zbMATH: 1493.62443
Digital Object Identifier: 10.1214/21-EJS1952

Primary: 62J07 , 62J99
Secondary: 62G08

Keywords: Aggregation , cross-validation , hyperparameter selection , Lasso , Model selection , robust regression , Sparse regression

Vol.16 • No. 1 • 2022
Back to Top