Classifiers of support vector machine type with \ell1 complexity regularization

Abstract

We study the binary classification problem with hinge loss. We consider classifiers that are linear combinations of base functions. Instead of an ${\ell }_2$ penalty, which is used by the support vector machine, we put an ${\ell }_1$ penalty on the coefficients. Under certain conditions on the base functions, hinge loss with this complexity penalty is shown to lead to an oracle inequality involving both model complexity and margin.