Open Access
August 2016 Performance of empirical risk minimization in linear aggregation
Guillaume Lecué, Shahar Mendelson
Bernoulli 22(3): 1520-1534 (August 2016). DOI: 10.3150/15-BEJ701


We study conditions under which, given a dictionary $F=\{f_{1},\ldots,f_{M}\}$ and an i.i.d. sample $(X_{i},Y_{i})_{i=1}^{N}$, the empirical minimizer in $\operatorname{span}(F)$ relative to the squared loss, satisfies that with high probability

\[R(\tilde{f}^{\mathrm{ERM}})\leq\inf_{f\in\operatorname{span}(F)}R(f)+r_{N}(M),\] where $R(\cdot)$ is the squared risk and $r_{N}(M)$ is of the order of $M/N$.

Among other results, we prove that a uniform small-ball estimate for functions in $\operatorname{span}(F)$ is enough to achieve that goal when the noise is independent of the design.


Download Citation

Guillaume Lecué. Shahar Mendelson. "Performance of empirical risk minimization in linear aggregation." Bernoulli 22 (3) 1520 - 1534, August 2016.


Received: 1 March 2014; Revised: 1 February 2015; Published: August 2016
First available in Project Euclid: 16 March 2016

zbMATH: 1346.60075
MathSciNet: MR3474824
Digital Object Identifier: 10.3150/15-BEJ701

Keywords: Aggregation theory , Empirical processes , empirical risk minimization , Learning theory

Rights: Copyright © 2016 Bernoulli Society for Mathematical Statistics and Probability

Vol.22 • No. 3 • August 2016
Back to Top