Functional aggregation for nonparametric regression

Abstract

We consider the problem of estimating an unknown function $f$ from $N$ noisy observations on a random grid. In this paper we address the following aggregation problem: given $M$ functions $f_1,\dots, f_M$, find an “aggregated ”estimator which approximates $f$ nearly as well as the best convex combination $f^*$ of $f_1,\dots, f_M$. We propose algorithms which provide approximations of $f^*$ with expected $L_2$ accuracy $O(N^{-1/4}\ln^{1/4} M$. We show that this approximation rate cannot be significantly improved. We discuss two specific applications: nonparametric prediction for a dynamic system with output nonlinearity and reconstruction in the Jones – Barron class.