The Annals of Statistics

Locally parametric nonparametric density estimation

N. L. Hjort and M. C. Jones

Full-text: Open access

Abstract

This paper develops a nonparametric density estimator with parametric overtones. Suppose $f(x, \theta)$ is some family of densities, indexed by a vector of parameters $\theta$. We define a local kernel-smoothed likelihood function which, for each x, can be used to estimate the best local parametric approximant to the true density. This leads to a new density estimator of the form $f(x, \hat{\theta}(x))$, thus inserting the best local parameter estimate for each new value of x. When the bandwidth used is large, this amounts to ordinary full likelihood parametric density estimation, while for moderate and small bandwidths the method is essentially nonparametric, using only local properties of data and the model. Alternative ways more general than via the local likelihood are also described. The methods can be seen as ways of nonparametrically smoothing the parameter within a parametric class.

Properties of this new semiparametric estimator are investigated. Our preferred version has approximately the same variance as the ordinary kernel method but potentially a smaller bias. The new method is seen to perform better than the traditional kernel method in a broad nonparametric vicinity of the parametric model employed, while at the same time being capable of not losing much in precision to full likelihood methods when the model is correct. Other versions of the method are approximately equivalent to using particular higher order kernels in a semiparametric framework. The methodology we develop can be seen as the density estimation parallel to local likelihood and local weighted least squares theory in nonparametric regression.

Article information

Source
Ann. Statist. Volume 24, Number 4 (1996), 1619-1647.

Dates
First available in Project Euclid: 17 September 2002

Permanent link to this document
http://projecteuclid.org/euclid.aos/1032298288

Digital Object Identifier
doi:10.1214/aos/1032298288

Mathematical Reviews number (MathSciNet)
MR1416653

Zentralblatt MATH identifier
0867.62030

Subjects
Primary: 62G07: Density estimation
Secondary: 62G20: Asymptotic properties

Keywords
Bias reduction density estimation kernel smoothing local likelihood local modelling parameter smoothing semiparametric estimation

Citation

Hjort, N. L.; Jones, M. C. Locally parametric nonparametric density estimation. Ann. Statist. 24 (1996), no. 4, 1619--1647. doi:10.1214/aos/1032298288. http://projecteuclid.org/euclid.aos/1032298288.


Export citation

References

  • Buckland, S. T. (1992). Maximum likelihood fitting of Hermite and simple poly nomial densities. J. Roy. Statist. Soc. Ser. C 41 241-266.
  • Cleveland, W. S. (1979). Robust locally weighted regression and smoothing scatterplots. J. Amer. Statist. Assoc. 74 829-836.
  • Copas, J. B. (1995). Local likelihood based on kernel censoring. J. Roy. Statist. Soc. Ser. B 57 221-235.
  • Efron, B. and Tibshirani, R. (1996). Using specially designed exponential families for density estimation. J. Amer. Statist. Assoc. To appear.
  • Fan, J. (1992). Design-adaptive nonparametric regression. J. Amer. Statist. Assoc. 87 998-1004.
  • Fan, J. (1993). Local linear regression smoothers and their minimax efficiencies. Ann. Statist. 21 196-216.
  • Fan, J. and Gijbels, I. (1992). Variable bandwidth and local linear regression smoothers. Ann. Statist. 20 2008-2036.
  • Fan, J. and Gijbels, I. (1996). Local Poly nomial Modelling and its Applications. Chapman and Hall, London.
  • Fan, J., Heckman, N. E. and Wand, M. P. (1995). Local poly nomial kernel regression for generalized linear models and quasi-likelihood functions. J. Amer. Statist. Assoc. 90 141-150.
  • Fenstad, G. U. and Hjort, N. L. (1996). Two Hermite expansion density estimators, and a comparison with the kernel method. Unpublished manuscript.
  • Hastie, T. and Loader, C. R. (1993). Local regression: Automatic kernel carpentry (with discussion). Statist. Sci. 8 120-143.
  • Hastie, T. and Tibshirani, R. (1990). Generalized Additive Models. Chapman and Hall, London.
  • Hjort, N. L. (1986). Theory of Statistical Sy mbol Recognition. Norwegian Computing Centre, Oslo.
  • Hjort, N. L. (1991). Semiparametric estimation of parametric hazard rates. In Survival Analy sis: State of the Art (P. S. Goel and J. P. Klein, eds.) 211-236. Kluwer, Dordrecht.
  • Hjort, N. L. (1994). Minimum L2 and robust Kullback-Leibler estimation. In Proceedings of the 12th Prague Conference on Information Theory, Statistical Decision Functions and Random Processes (P. Lachout and J. ´A. V´i sek, eds.) 102-105. Academy of Sciences of the Czech Republic, Prague.
  • Hjort, N. L. (1995). Bayesian approaches to semiparametric density estimation (with discussion). In Bayesian Statistics V (J. Bernardo, J. Berger, P. Dawid and A. F. M. Smith, eds.) 223-253. Oxford Univ. Press. Hjort, N. L. (1996a). Performance of Efron and Tibshirani's semiparametric density estimator. Statistical research report, Dept. Mathematics, Univ. Oslo. Hjort, N. L. (1996b). Multiplicative higher order bias kernel density estimators. Statistical research report, Dept. Mathematics, Univ. Oslo.
  • Hjort, N. L. (1997). Dy namic likelihood hazard rate estimation. Biometrika 84. To appear.
  • Hjort, N. L. and Glad, I. K. (1995). Nonparametric density estimation with a parametric start. Ann. Statist. 23 882-904.
  • Hjort, N. L. and Pollard, D. B. (1996). Asy mptotics for minimizers of convex processes. Unpublished manuscript. Jones, M. C. (1993a). Kernel density estimation when the bandwidth is large. Austral. J. Statist. 35 319-326. Jones, M. C. (1993b). Simple boundary correction for kernel density estimation. Statistics and Computing 3 135-146.
  • Jones, M. C. (1994). On kernel density derivative estimation. Comm. Statist. Theory Methods 23 2133-2139.
  • Jones, M. C. (1995). On close relations of local likelihood density estimation. Unpublished manuscript.
  • Jones, M. C., Davies, S. J. and Park, B. U. (1994). Versions of kernel-ty pe regression estimators. J. Amer. Statist. Assoc. 89 825-832.
  • Jones, M. C. and Foster, P. J. (1993). Generalized jackknifing and higher order kernels. J. Nonparametr. Statist. 3 81-94.
  • Jones, M. C. and Hjort, N. L. (1994). Local fitting of regression models by likelihood: what's important? Statistical research report, Dept. Mathematics, Univ. Oslo.
  • Jones, M. C., Linton, O. and Nielsen, J. P. (1995). A simple and effective bias reduction method for density and regression estimation. Biometrika 82 327-338.
  • Jones, M. C., Marron, J. S. and Sheather, S. J. (1996). A brief survey of bandwidth selection for density estimation. J. Amer. Statist. Assoc. 91 401-407.
  • Lindsey, J. K. (1974). Comparison of probability distributions. J. Roy. Statist. Soc. Ser. B 36 38-47.
  • Loader, C. R. (1996). Local likelihood density estimation. Ann. Statist. 24 1602-1618.
  • Marron, J. S. and Wand, M. P. (1992). Exact mean integrated squared error. Ann. Statist. 20 712-736.
  • Olkin, I. and Spiegelman, C. H. (1987). A semiparametric approach to density estimation. J. Amer. Statist. Assoc. 82 858-865.
  • Ruppert, D. and Wand, M. P. (1994). Multivariate locally weighted least squares regression. Ann. Statist. 22 1346-1370.
  • Schuster, E. and Yakowitz, S. (1985). Parametric/nonparametric mixture density estimation with application to flood-frequency analysis. Water Resources Bulletin 21 797-804.
  • Scott, D. W. (1992). Multivariate Density Estimation: Theory, Practice, and Visualization. Wiley, New York.
  • Shao, J. (1991). Second-order differentiability and jackknife. Statist. Sinica 1 185-202.
  • Sheather, S. J. and Jones, M. C. (1991). A reliable data-based bandwidth selection method for kernel density estimation. J. Roy. Statist. Assoc. B 53 683-690.
  • Staniswalis, J. (1989). The kernel estimate of a regression function in likelihood-based models. J. Amer. Statist. Assoc. 84 276-283.
  • Stone, C. J. (1977). Consistent nonparametric regression. Ann. Statist. 5 595-620.
  • Tibshirani, R. and Hastie, T. (1987). Local likelihood estimation. J. Amer. Statist. Assoc. 82 559-567.
  • Wand, M. P. and Jones, M. C. (1993). Comparison of smoothing parameterizations in bivariate kernel density estimation. J. Amer. Statist. Assoc. 88 520-528.
  • Wand, M. P. and Jones, M. C. (1995). Kernel Smoothing. Chapman and Hall, London.