Nonparametric Maximum Likelihood Estimation of Probability Densities by Penalty Function Methods

Abstract

Tne maximum likelihood estimate of a probability density function based on a random sample does not exist in the nonparametric case. For this reason and others based on heuristic Bayesian considerations Good and Gaskins suggested adding a penalty term to the likelihood. They proposed two penalty terms; however they did not establish existence or uniqueness of their maximum penalized likelihood estimates. Good and Gaskins also suggested an alternate approach for calculating the maximum penalized likelihood estimate which avoids the nonnegativity constraint on the estimate. In the present work the existence and uniqueness of both of Good's and Gaskins' maximum penalized likelihood estimates are rigorously demonstrated. Moreover, it is shown that one of these estimates is a positive exponential spline with knots only at the sample points and that in this case the alternate approach leads to the correct estimate; however in the other case the alternate approach leads to the wrong estimate. Finally, it is shown that a well-known class of reproducing kernel Hilbert spaces leads very naturally to maximum penalized likelihood estimates which are polynomial splines with knots at the sample points.