Bias Reduction in Kernel Density Estimation by Smoothed Empirical Transformations

Abstract

A modification of kernel density estimation is proposed. The first step is ordinary kernel estimation of the density and its cdf. In the second step the data are transformed, using this estimated cdf, to an approximate uniform (or normal or other target) distribution. The density and cdf of the transformed data are then estimated by the kernel method and, by change of variable, converted to new estimates of the density and the cdf of the original data. This process is repeated for a total of $k$ steps for some integer $k$ greater than 1. If the target density is uniform, then the order of the bias is reduced, provided that the density of the observed data is sufficiently smooth. By proper choice of bandwidth, rates of squared-error convergence equal to those of higher-order kernels are attainable. More precisely, $k$ repetitions of the process are equivalent, in terms of rate of convergence, to a $2k$-th-order kernel. This transformation-kernel estimate is always a bona fide density and appears to be more effective at small sample sizes than higher-order kernel estimators, at least for densities with interesting features such as multiple modes. The main theoretical achievement of this paper is the rigorous establishment of rates of convergence under multiple iteration. Simulations using a uniform target distribution suggest that the possibility of improvement over ordinary kernel estimation is of practical significance for samples sizes as low as 100 and can become appreciable for sample sizes around 400.