Open Access
Translator Disclaimer
2018 Fast learning rate of non-sparse multiple kernel learning and optimal regularization strategies
Taiji Suzuki
Electron. J. Statist. 12(2): 2141-2192 (2018). DOI: 10.1214/18-EJS1399


In this paper, we give a new generalization error bound of Multiple Kernel Learning (MKL) for a general class of regularizations, and discuss what kind of regularization gives a favorable predictive accuracy. Our main target in this paper is dense type regularizations including $\ell_{p}$-MKL. According to the numerical experiments, it is known that the sparse regularization does not necessarily show a good performance compared with dense type regularizations. Motivated by this fact, this paper gives a general theoretical tool to derive fast learning rates of MKL that is applicable to arbitrary mixed-norm-type regularizations in a unifying manner. This enables us to compare the generalization performances of various types of regularizations. As a consequence, we observe that the homogeneity of the complexities of candidate reproducing kernel Hilbert spaces (RKHSs) affects which regularization strategy ($\ell_{1}$ or dense) is preferred. In fact, in homogeneous complexity settings where the complexities of all RKHSs are evenly same, $\ell_{1}$-regularization is optimal among all isotropic norms. On the other hand, in inhomogeneous complexity settings, dense type regularizations can show better learning rate than sparse $\ell_{1}$-regularization. We also show that our learning rate achieves the minimax lower bound in homogeneous complexity settings.


Download Citation

Taiji Suzuki. "Fast learning rate of non-sparse multiple kernel learning and optimal regularization strategies." Electron. J. Statist. 12 (2) 2141 - 2192, 2018.


Received: 1 March 2017; Published: 2018
First available in Project Euclid: 13 July 2018

zbMATH: 06917472
MathSciNet: MR3827817
Digital Object Identifier: 10.1214/18-EJS1399

Keywords: fast learning rate , generalization error bounds , mini-max lower bound , multiple kernel learning , regularization


Vol.12 • No. 2 • 2018
Back to Top