Translator Disclaimer
2017 Kernel ridge vs. principal component regression: Minimax bounds and the qualification of regularization operators
Lee H. Dicker, Dean P. Foster, Daniel Hsu
Electron. J. Statist. 11(1): 1022-1047 (2017). DOI: 10.1214/17-EJS1258

Abstract

Regularization is an essential element of virtually all kernel methods for nonparametric regression problems. A critical factor in the effectiveness of a given kernel method is the type of regularization that is employed. This article compares and contrasts members from a general class of regularization techniques, which notably includes ridge regression and principal component regression. We derive an explicit finite-sample risk bound for regularization-based estimators that simultaneously accounts for (i) the structure of the ambient function space, (ii) the regularity of the true regression function, and (iii) the adaptability (or qualification) of the regularization. A simple consequence of this upper bound is that the risk of the regularization-based estimators matches the minimax rate in a variety of settings. The general bound also illustrates how some regularization techniques are more adaptable than others to favorable regularity properties that the true regression function may possess. This, in particular, demonstrates a striking difference between kernel ridge regression and kernel principal component regression. Our theoretical results are supported by numerical experiments.

Citation

Download Citation

Lee H. Dicker. Dean P. Foster. Daniel Hsu. "Kernel ridge vs. principal component regression: Minimax bounds and the qualification of regularization operators." Electron. J. Statist. 11 (1) 1022 - 1047, 2017. https://doi.org/10.1214/17-EJS1258

Information

Received: 1 August 2016; Published: 2017
First available in Project Euclid: 30 March 2017

zbMATH: 1362.62087
MathSciNet: MR3629418
Digital Object Identifier: 10.1214/17-EJS1258

Subjects:
Primary: 62G08

JOURNAL ARTICLE
26 PAGES


SHARE
Vol.11 • No. 1 • 2017
Back to Top