Optimal Convergence Properties of Variable Knot, Kernel, and Orthogonal Series Methods for Density Estimation

Abstract

Let $W_p^{(m)}(M) = \{f: f^{(\nu)} \operatorname{abs}. \operatorname{cont}., \nu = 0, 1,\cdots, m - 1, f^{(m)} \in \mathscr{L}_p, \|f^{(m)}\|_p \leqq M\}$, where $\|\cdot\|_p$ is the norm in $\mathscr{L}_p, m$ is a positive integer and $p$ is a real number, $p \geqq 1$. Let $\{\hat{f}_n(x)\}, n = 1, 2,\cdots$ be any sequence of estimates of a density at the point $x$ where $\hat{f}_n(x)$ depends on $n$ independent observations from some density $f \in W_p^{(m)}(M)$. It is shown that if $\sup_{f\in W_p^{(m)} (M)} E_f(f(x) - \hat{f}_n(x))^2 = b_nn^{-\phi(m, p+\varepsilon)}$, where $\phi(m, p) = (2m - 2/p)/(2m + 1 - 2/p)$, and $\varepsilon > 0$, then there exists a $D_0 > 0$ such that $b_n \geqq D_0$ for infinitely many $n$. Thus the best possible mean square convergence rate for a density estimate, which is uniform over $W_p^{(m)}(M)$, is not better than $n^{-\phi(m,p+\varepsilon)}$ for arbitrarily small $\varepsilon$. The following types of density estimates are shown to have mean square error at a point bounded above by $Dn^{-\phi(m,p)}$, provided that a certain parameter, usually depending on $m, p$ and $M$, is chosen optimally: the polynomial algorithm, kernel-type estimates, certain orthogonal series estimates, and the ordinary histogram. $D$'s for each method are given.