A Comparison between Fixed-Basis and Variable-Basis Schemes for Function Approximation and Functional Optimization

Abstract

Fixed-basis and variable-basis approximation schemes are compared for the problems of function approximation and functional optimization (also known as infinite programming). Classes of problems are investigated for which variable-basis schemes with sigmoidal computational units perform better than fixed-basis ones, in terms of the minimum number of computational units needed to achieve a desired error in function approximation or approximate optimization. Previously known bounds on the accuracy are extended, with better rates, to families of $d$-variable functions whose actual dependence is on a subset of $d^{\prime }\ll d$ variables, where the indices of these $d^{\prime }$ variables are not known a priori.