Open Access
Translator Disclaimer
2022 Aggregated hold out for sparse linear regression with a robust loss function
Guillaume Maillard
Author Affiliations +
Electron. J. Statist. 16(1): 935-997 (2022). DOI: 10.1214/21-EJS1952

Abstract

Sparse linear regression methods generally have a free hyperparameter which controls the amount of sparsity, and is subject to a bias-variance tradeoff. This article considers the use of Aggregated hold-out to aggregate over values of this hyperparameter, in the context of linear regression with the Huber loss function. Aggregated hold-out (Agghoo) is a procedure which averages estimators selected by hold-out (cross-validation with a single split). In the theoretical part of the article, it is proved that Agghoo satisfies a non-asymptotic oracle inequality when it is applied to sparse estimators which are parametrized by their zero-norm. In particular, this includes a variant of the Lasso introduced by Zou, Hastié and Tibshirani [49]. Simulations are used to compare Agghoo with cross-validation. They show that Agghoo performs better than CV when the intrinsic dimension is high and when there are confounders correlated with the predictive covariates.

Funding Statement

While finishing the writing of this article, the author (Guillaume Maillard) has received funding from the European Union’s Horizon 2020 research and innovation program under grant agreement No 811017.

Citation

Download Citation

Guillaume Maillard. "Aggregated hold out for sparse linear regression with a robust loss function." Electron. J. Statist. 16 (1) 935 - 997, 2022. https://doi.org/10.1214/21-EJS1952

Information

Received: 1 February 2020; Published: 2022
First available in Project Euclid: 3 February 2022

Digital Object Identifier: 10.1214/21-EJS1952

Subjects:
Primary: 62J07 , 62J99
Secondary: 62G08

Keywords: Aggregation , cross-validation , hyperparameter selection , Lasso , Model selection , robust regression , Sparse regression

JOURNAL ARTICLE
63 PAGES


SHARE
Vol.16 • No. 1 • 2022
Back to Top