Approximation Analysis of Learning Algorithms for Support Vector Regression and Quantile Regression

Abstract

We study learning algorithms generated by regularization schemes in reproducing kernel Hilbert spaces associated with an $\mathrm{\{\backslash epsilon\}}$-insensitive pinball loss. This loss function is motivated by the $\mathrm{\{\backslash epsilon\}}$-insensitive loss for support vector regression and the pinball loss for quantile regression. Approximation analysis is conducted for these algorithms by means of a variance-expectation bound when a noise condition is satisfied for the underlying probability measure. The rates are explicitly derived under a priori conditions on approximation and capacity of the reproducing kernel Hilbert space. As an application, we get approximation orders for the support vector regression and the quantile regularized regression.