The Annals of Statistics

Maximum likelihood estimation of smooth monotone and unimodal densities

P. P. B. Eggermont and V. N. LaRiccia

Source: Ann. Statist. Volume 28, Number 3 (2000), 922-947.

Abstract

We study the nonparametric estimation of univariate monotone and unimodal densities usingthe maximum smoothed likelihood approach. The monotone estimator is the derivative of the least concave majorant of the distribution correspondingto a kernel estimator.We prove that the mapping on distributions $\Phi$ with density $\varphi$,

$$\varphi \mapsto \text{the derivative of the least concave majorant of $\Phi},$$

is a contraction in all $L^P$ norms $(1 \leq p \leq \infty)$, and some other “distances” such as the Hellinger and Kullback–Leibler distances. The contractivity implies error bounds for monotone density estimation. Almost the same error bounds hold for unimodal estimation.

Primary Subjects: 62G07
Keywords: Maximum likelihood estimation; monotone and unimodal densities; least concave majorants; contractions; $L^1$ error bounds

Full-text: Open access

Links and Identifiers

Permanent link to this document: http://projecteuclid.org/euclid.aos/1015952005
Mathematical Reviews number (MathSciNet): MR1792794
Digital Object Identifier: doi:10.1214/aos/1015952005
Zentralblatt MATH identifier: 01828969

References

Barlow, R. E., Bartholomew, D. J., Bremner, J. M. and Brunk, H. D. (1972). Statistical Inference under Order Restrictions. Wiley, New York.
Bickel, P. J. and Fan, I. (1996). Some problems on the estimation of unimodal densities. Statist. Sinica 6 23-45. Birg´e, L. (1987a). Estimatinga density under restrictions: nonasymptotic minimax risk. Ann. Statist. 15 995-1012. Birg´e, L. (1987b). On the risk of histograms for estimatingdecreasingdensities. Ann. Statist. 15 1113-1022.
Birg´e, L. (1989). The Grenander estimator: a nonasymptotic approach. Ann. Statist. 17 1532-1549.
Mathematical Reviews (MathSciNet): MR91d:62035
Birg´e, L. (1997). Estimation of unimodal densities without smoothness assumptions. Ann. Statist. 25 970-981.
Mathematical Reviews (MathSciNet): MR98g:62056
Brunk, H. D. (1965). Conditional expectation given a lattice and applications. Ann. Math. Statist. 36 1339-1350.
Mathematical Reviews (MathSciNet): MR32:3091
Chernoff, H. (1964). Estimation of the mode. Ann. Inst. Statist. Math. 16 31-41.
Mathematical Reviews (MathSciNet): MR30:2601
Devroye, L. (1987). A Course in Density Estimation. Birkh¨auser, Boston.
Mathematical Reviews (MathSciNet): MR88d:62070
Devroye, L. (1991). Exponential inequalities in nonparametric estimation. In Nonparametric Functional Estimation and Related Topics (G. Roussas, ed.) 31-44. Kluwer, Dordrecht.
Mathematical Reviews (MathSciNet): MR93c:62066
Devroye, L. and Gy ¨orfi, L. (1985). Density Estimation: the L1-View. Wiley, New York.
Devroye, L. Gy ¨orfi, L. and Lugosi, G. (1996). A Probabilistic Theory of Pattern Recognition. Springer, New York.
Mathematical Reviews (MathSciNet): MR97d:68196
Eddy, W. F. (1980). Optimal kernel estimators of the mode. Ann. Statist. 8 870-882.
Eggermont, P. P. B. and LaRiccia, V. N. (1995). Maximum smoothed likelihood density estimation. J. Nonparametr. Statist. 4 211-222.
Zentralblatt MATH: 0822.62025
Eggermont, P. P. B. and LaRiccia, V. N. (1999). Best asymptotically normal kernel density entropy estimators for smooth densities. IEEE Trans Inform. Theory 45 1321-1326.
Mathematical Reviews (MathSciNet): MR2000d:62050
Foug eres, A.-L. (1997). Estimation de densit´es unimodales. Canad. J. Statist. 25 375-387.
Mathematical Reviews (MathSciNet): MR99c:62088
Grenander, U. (1956). On the theory of mortality measurements II. Skand. Akt. 39 125-153.
Groeneboom, P. (1985). Estimatinga monotone density. In Proceedings of the Berkeley Conference in Honor of Jerzy Neyman and Jack Kiefer (L. M. Le Cam and R. A. Olshen, eds.) 539-555. Wadsworth, Belmont, CA.
Mathematical Reviews (MathSciNet): MR87i:62076
Groeneboom, P., Hooghiemstra, G. and Lopuha¨a, H. P. (1999). Asymptotic normality of the L1 error of the Grenander estimator. Ann. Statist. 27 1316-1347.
Mathematical Reviews (MathSciNet): MR2001c:62016
Grund, B. and Hall, P. (1995). On the minimisation of Lp error in mode estimation. Ann. Statist. 23 2264-2284.
Mathematical Reviews (MathSciNet): MR97c:62074
Hall, P. and Wehrly, T. E. (1991). A geometrical method for removing edge effects from kerneltype nonparametric regression estimators. J. Amer. Statist. Assoc. 86 665-672.
Hardy, G. H., Littlewood, G. E. and P ´olya, G. (1951). Inequalities. Cambridge Univ. Press.
Ibragimov, I. A. (1956). On the composition of unimodal distributions. Theory Probab. Appl. 1 255-260.
Zentralblatt MATH: 0073.12501
Jones, M. C. (1993). Simple boundary corrections for kernel density estimation. Statist. Comput. 3 135-146.
Kemperman, J. H. B. (1967). On the optimum rate of transmittinginformation. Lecture Notes in Math. 23 126-169. Springer, Berlin.
Lieb, E. H. and Loss, M. (1996). Analysis. Amer. Math. Soc., Providence, RI.
Mammen, E. (1991). Estimatinga smooth monotone regression function. Ann. Statist. 19 724-740.
Mathematical Reviews (MathSciNet): MR92g:62042
Marshall, A. W. (1970). Discussion on Barlow and van Zwet's paper. In Nonparametric Techniques in Statistical Inference (M. L. Puri, ed.) 175-176. Cambridge Univ. Press.
M ¨uller, H.-G. (1993). On the boundary kernel method for nonparametric curve estimation near endpoints. Scand. J. Statist. 20 313-328.
Parzen, E. (1962). On estimation of a probability density function and mode. Ann. Math. Statist. 33 1065-1076.
Mathematical Reviews (MathSciNet): MR26:841
Silverman, B. W. (1978). Weak and stronguniform consistency of the kernel estimate of a density and its derivatives. Ann. Statist. 6 177-184.
Mathematical Reviews (MathSciNet): MR57:10904
Silverman, B. W. (1982). On the estimation of a probability density function by the maximum penalized likelihood method. Ann. Statist. 10 795-810.
Mathematical Reviews (MathSciNet): MR84c:62059
Wegman, E. I. (1970). Maximum likelihood estimation of a unimodal density II. Ann. Math. Statist. 6 2169-2174.
Mathematical Reviews (MathSciNet): MR42:2583
Wheeden, R. and Zygmund, A. (1977). Measure and Integral. North-Holland, Amsterdam.
Mathematical Reviews (MathSciNet): MR58:11295
Woodroofe, M. and Sun, J. (1993). A penalized maximum likelihood estimate of f 0+ when f is non-increasing Statist. Sinica 3 501-515.
Mathematical Reviews (MathSciNet): MR94i:62044

2009 © Institute of Mathematical Statistics