Estimation of a Projection-Pursuit Type Regression Model

Abstract

Since the pioneering work of Friedman and Stuetzle in 1981, projection-pursuit algorithms have attracted increasing attention. This is mainly due to their potential for overcoming or reducing difficulties arising in nonparametric regression models associated with the so-called curse of dimensionality, that is, the amount of data required to avoid an unacceptably large variance increasing rapidly with dimensionality. Subsequent work has, however, uncovered a dependence on dimensionality for projection-pursuit regression models. Here we propose a projection-pursuit type estimation scheme, with two additional constraints imposed, for which the rate of convergence of the estimator is shown to be independent of the dimensionality. Let $(\mathbf{X}, Y)$ be a random vector such that $\mathbf{X} = (X_1, \ldots, X_d)^T$ ranges over $R^d$. The conditional mean of $Y$ given $\mathbf{X} = \mathbf{x}$ is assumed to be the sum of no more than $d$ general smooth functions of $\beta^T_i\mathbf{x}$, where $\beta_i \in S^{d - 1}$, the unit sphere in $R^d$ centered at the origin. A least-squares polynomial spline and the final prediction error criterion are used to fit the model to a random sample of size $n$ from the distribution of $(\mathbf{X}, Y)$. Under appropriate conditions, the rate of convergence of the proposed estimator is independent of $d$.