Density Problem and Approximation Error in Learning Theory

Abstract

We study the density problem and approximation error of reproducing kernel Hilbert spaces for the purpose of learning theory. For a Mercer kernel $K$ on a compact metric space ($X$, $d$), a characterization for the generated reproducing kernel Hilbert space (RKHS) ${\mathscr{H}}_K$ to be dense in $\mathrm{C(X)}$ is given. As a corollary, we show that the density is always true for convolution type kernels. Some estimates for the rate of convergence of interpolation schemes are presented for general Mercer kernels. These are then used to establish for convolution type kernels quantitative analysis for the approximation error in learning theory. Finally, we show by the example of Gaussian kernels with varying variances that the approximation error can be improved when we adaptively change the value of the parameter for the used kernel. This confirms the method of choosing varying parameters which is used often in many applications of learning theory.