An Adaptive Orthogonal-Series Estimator for Probability Density Functions

Abstract

Given a sample set $X_1, \cdots, X_N$ of independent identically distributed real-valued random variables, each with the unknown probability density function $f(\cdot)$, the problem considered is to estimate $f$ from the sample set. The function $f$ is assumed to be in $L_2(a, b); f$ is not assumed to be in any parametric family. This paper constructs an adaptive "two-pass" solution to the problem: in a preprocessing step (the first pass), a preliminary rough estimate of $f$ is obtained by means of a standard orthogonal-series estimator. In the second pass, the preliminary estimate is used to transform the orthogonal series. The new, transformed orthogonal series is then used to obtain the final estimate. The paper establishes consistency of the estimator and derives asymptotic (large sample set) estimates of the bias and variance. It is shown that the adaptive estimator offers reduced bias (better resolution) in comparison to the conventional orthogonal series estimator. Computer simulations are presented which demonstrate the small sample set behavior. A case study of a bimodal density confirms the theoretical conclusions.