A New Approach to Least-Squares Estimation, with Applications

Abstract

The regression model $\mathbf{y} = g(\mathbf{x}) + \mathbf{\varepsilon}$ and least-squares estimation are studied in a general context. By making use of empirical process theory, it is shown that entropy conditions on the class $\mathscr{G}$ of possible regression functions imply $L^2$-consistency of the least-squares estimator $\hat{\mathbf{g}}_n$ of $g$. This result is applied in parametric and nonparametric regression.