We study the asymptotic behavior of the least squares estimators when the model is possibly misspecified. We consider the setting where we wish to estimate an unknown function $f_{*}:(0,1)^{d}\rightarrow \mathbb{R}$ from observations $(X,Y),(X_{1},Y_{1}),\cdots ,(X_{n},Y_{n})$; our estimator $\hat{g}_{n}$ is the minimizer of $\sum _{i=1}^{n}(Y_{i}-g(X_{i}))^{2}/n$ over $g\in \mathcal{G}$ for some set of functions $\mathcal{G}$. We provide sufficient conditions on the metric entropy of $\mathcal{G}$, under which $\hat{g}_{n}$ converges to $g_{*}$ as $n\rightarrow \infty $, where $g_{*}$ is the minimizer of $\|g-f_{*}\|\triangleq \mathbb{E}(g(X)-f_{*}(X))^{2}$ over $g\in \mathcal{G}$. As corollaries of our theorem, we establish $\|\hat{g}_{n}-g_{*}\|\rightarrow 0$ as $n\rightarrow \infty $ when $\mathcal{G}$ is the set of monotone functions or the set of convex functions. We also make a connection between the convergence rate of $\|\hat{g}_{n}-g_{*}\|$ and the metric entropy of $\mathcal{G}$. As special cases of our finding, we compute the convergence rate of $\|\hat{g}_{n}-g_{*}\|^{2}$ when $\mathcal{G}$ is the set of bounded monotone functions or the set of bounded convex functions.
Electron. J. Statist.
14(1):
2053-2097
(2020).
DOI: 10.1214/20-EJS1714