Optimal Estimation of a General Regression Function

Abstract

Let $\Theta$ be a convex subset of $R^d$, and for each $\theta \in \Theta$ let $F(\theta; dt)$ be a probability on the line. For any vector $a_n = (a_{n1}, a_{n2}, \cdots, a_{nn})$ where $a_{ni} \in \Theta$, let $X_{n1}, \cdots, X_{nn}$ be independent observations, the distribution of $X_{ni}$ being $F(a_{ni}; dt)$. The main results give a method of estimating the unknown regression function $a_n$ based on a minimum distance recipe. Under regularity assumptions, the proposed estimators are shown, in an appropriate framework, to be asymptotically normal, locally asymptotically minimax, and robust. The abstract results are illustrated by application to the linear model and to exponential response models. In general, nothing at all is assumed about the form of the regression function; accordingly, this forces the limiting normal distributions of the proposed estimators to be located on infinite dimensional linear spaces.