Estimation of Probability Density by an Orthogonal Series

Abstract

Let $X_1 \cdots X_n$ represent a sequence of independent random variables with a common (unknown) density function $f(x)$. In this paper, an estimate of $f(x)$ of the form $\hat{f}_n(x) = \sum^{q(n)}_{j = 0} \hat{a}_{jn\varphi_j}(x)$ is considered, where $\hat{a}_{jn} = (1/n) \sum^n_{i = 1\varphi_j}(X_i), \varphi_j( )$ is the $j$th Hermite function and $q(n)$ is an integer dependent on $n$. Assuming $f(x)$ is $L_2$, it is shown that the sequence of estimates is consistent in the sense of mean integrated square error, $\lim_{n \rightarrow \infty} E \cdot \int (f(x) - \hat f_n(x))^2 dx = 0$ and, under additional conditions on $f(x)$, the sequence of estimates is also consistent in mean square error, $\lim_{n \rightarrow \infty} E(f(x) - \hat{f}_n(x))^2 = 0$, uniformly in $x$. For both error criteria, bounds on the rate of convergence of the estimate are obtained. The rate of convergence is seen to depend on the smoothness and integrability properties of $f(x)$--the maximum rate being bounded by $1/n$. In order for the series method to achieve the same rate of convergence as an estimate which uses the "kernel" technique [4], [6], more assumptions on $f(x)$ are required. However, in estimating a multivariate density, with the same type of conditions as in the univariate case, the rate of convergence remains the same for the multivariate series estimate. With the "kernel" method, the rate depends on the dimension of the density being estimated; the rate of convergence of the estimate decreases as the dimension increases. In the next section, we introduce notation and give some preliminary results. Conditions for consistency and rates of convergence are established in Section 3. These results are then compared in Section 4 to previous work in the area.