Open Access
Translator Disclaimer
2009 Model selection by resampling penalization
Sylvain Arlot
Electron. J. Statist. 3: 557-624 (2009). DOI: 10.1214/08-EJS196


In this paper, a new family of resampling-based penalization procedures for model selection is defined in a general framework. It generalizes several methods, including Efron’s bootstrap penalization and the leave-one-out penalization recently proposed by Arlot (2008), to any exchangeable weighted bootstrap resampling scheme. In the heteroscedastic regression framework, assuming the models to have a particular structure, these resampling penalties are proved to satisfy a non-asymptotic oracle inequality with leading constant close to 1. In particular, they are asympotically optimal. Resampling penalties are used for defining an estimator adapting simultaneously to the smoothness of the regression function and to the heteroscedasticity of the noise. This is remarkable because resampling penalties are general-purpose devices, which have not been built specifically to handle heteroscedastic data. Hence, resampling penalties naturally adapt to heteroscedasticity. A simulation study shows that resampling penalties improve on V-fold cross-validation in terms of final prediction error, in particular when the signal-to-noise ratio is not large.


Download Citation

Sylvain Arlot. "Model selection by resampling penalization." Electron. J. Statist. 3 557 - 624, 2009.


Published: 2009
First available in Project Euclid: 19 June 2009

zbMATH: 1326.62097
MathSciNet: MR2519533
Digital Object Identifier: 10.1214/08-EJS196

Primary: 62G09
Secondary: 62G08 , 62M20

Keywords: Adaptivity , exchangeable weighted bootstrap , heteroscedastic data , histogram selection , Model selection , Non-parametric regression , Non-parametric statistics , Penalization , regressogram , Resampling

Rights: Copyright © 2009 The Institute of Mathematical Statistics and the Bernoulli Society


Back to Top