We prove oracle inequalities for three different types of adaptation procedures inspired by cross-validation and aggregation. These procedures are then applied to the construction of Lasso estimators and aggregation with exponential weights with data-driven regularization and temperature parameters, respectively. We also prove oracle inequalities for the cross-validation procedure itself under some convexity assumptions.
"Oracle inequalities for cross-validation type procedures." Electron. J. Statist. 6 1803 - 1837, 2012. https://doi.org/10.1214/12-EJS730