Electronic Journal of Statistics

Theoretical properties of the log-concave maximum likelihood estimator of a multidimensional density

Madeleine Cule and Richard Samworth

Full-text: Open access

Abstract

We present theoretical properties of the log-concave maximum likelihood estimator of a density based on an independent and identically distributed sample in ℝd. Our study covers both the case where the true underlying density is log-concave, and where this model is misspecified. We begin by showing that for a sequence of log-concave densities, convergence in distribution implies much stronger types of convergence – in particular, it implies convergence in Hellinger distance and even in certain exponentially weighted total variation norms. In our main result, we prove the existence and uniqueness of a log-concave density that minimises the Kullback–Leibler divergence from the true density over the class of all log-concave densities, and also show that the log-concave maximum likelihood estimator converges almost surely in these exponentially weighted total variation norms to this minimiser. In the case of a correctly specified model, this demonstrates a strong type of consistency for the estimator; in a misspecified model, it shows that the estimator converges to the log-concave density that is closest in the Kullback–Leibler sense to the true density.

Article information

Source
Electron. J. Statist. Volume 4 (2010), 254-270.

Dates
First available in Project Euclid: 17 February 2010

Permanent link to this document
http://projecteuclid.org/euclid.ejs/1266416850

Digital Object Identifier
doi:10.1214/09-EJS505

Mathematical Reviews number (MathSciNet)
MR2645484

Zentralblatt MATH identifier
06165736

Subjects
Primary: 62G07: Density estimation 62G20: Asymptotic properties

Keywords
Consistency log-concavity Kullback–Leibler divergence maximum likelihood estimation model misspecification

Citation

Cule, Madeleine; Samworth, Richard. Theoretical properties of the log-concave maximum likelihood estimator of a multidimensional density. Electron. J. Statist. 4 (2010), 254--270. doi:10.1214/09-EJS505. http://projecteuclid.org/euclid.ejs/1266416850.


Export citation

References

  • Bagnoli, M. and Bergstrom, T. (1989), Log-concave probability and its applications, Unpublished manuscript, University of, Michigan.
  • Balabdaoui, F., Jankowski, H. K., Pavlides, M., Seregin, A. and Wellner, J. A. (2010), On the Grenander estimator at zero, Statistica Sinica, to appear.
  • Balabdaoui, F., Rufibach, K. and Wellner, J. A. (2009), Limit distribution theory for maximum likelihood estimation of a log-concave density, Ann. Statist., 37, 1299–1331.
  • Balabdaoui, F. and Wellner, J. A. (2007), Estimation of a, k-monotone density: limit distribution theory and the spline connection, Ann. Statist., 35, 2536–2564.
  • Balabdaoui, F. and Wellner, J. A. (2010), Estimation of a k-monotone density: characterizations, consistency and minimax lower bounds, Statistica Neerlandica, 64, 45–70.
  • Billingsley, P. (1999), Convergence of Probability Measures, Wiley, New York.
  • Birgé, L. (1989), The Grenander estimator: a nonasymptotic approach, Ann. Statist., 17, 1532–1549.
  • Birgé, L. (1997), Estimation of unimodal densities without smoothness assumptions, Ann. Statist., 25, 970–981.
  • Bhattacharya, R. N. and Rao, R. R., Normal Approximation and Asymptotic Expansions, Wiley, New York.
  • Cule, M. L., Gramacy, R. B. and Samworth, R. J. (2007), LogConcDEAD: an, R package for log-concave density estimation in arbitrary dimensions. Available from http://cran.r-project.org/.
  • Cule, M. L., Gramacy, R. B. and Samworth, R. J. (2009) LogConcDEAD: an, R package for maximum likelihood estimation of a multivariate log-concave density, J. Statist. Software, 29, Issue 2.
  • Cule, M. L., Samworth, R. J. and Stewart, M. I. (2010), Maximum likelihood estimation of a multidimensional log-concave density, J. Roy. Statist. Soc., Ser. B (with discussion), to appear.
  • Devroye, L. (1987), A course in density estimation, Birkhäuser Boston Inc., Boston MA.
  • Dharmadhikari, S. and Joag-Dev, K. (1988), Unimodality, Convexity and Applications, Academic Press, Inc., San Diego, CA.
  • Dümbgen, L., Hüsler, A. and Rufibach, K. (2007), Active Set and EM Algorithms for Log-Concave Densities Based on Complete and Censored Data. Preprint., http://arxiv.org/abs/0707.4643.
  • Dümbgen, L. and Rufibach, K. (2009), Maximum likelihood estimation of a log-concave density and its distribution function: basic properties and uniform consistency, Bernoulli, 15, 40–68.
  • Dümbgen, L., Rufibach. K. and Wellner, J. A. (2007), Marshall’s lemma for convex density estimation, Asymptotics: particles, processes and inverse problems, 101–107, IMS Lecture Notes Monogr. Ser., 55, Inst. Math. Statist., Beachwood, OH.
  • Folland, G. B. (1999), Real Analysis, Wiley, New York.
  • Grenander, U. (1956), On the theory of mortality measurement. II., Skand. Aktuarietidskr., 39, 125–153.
  • Groeneboom, P., Jongbloed, G. and Wellner, J. A. (2001), Estimation of a convex function: Characterization and asymptotic theory, Ann. Statist., 29, 1653–1698.
  • Marshall, A. W. and Proschan, F. (1965), Maximum likelihood estimation for distributions with monotone failure rate, Ann. Math. Statist., 36, 69–77.
  • Pal, J., Woodroofe, M. and Meyer, M. (2007), Estimating a Polya frequency function. In, Complex datasets and Inverse problems: Tomography, Networks and Beyond (eds. R. Liu, W. Strawderman, C.-H. Zhang), pp. 239–249. IMS Lecture Notes - Monograph Series 54.
  • Prakasa Rao, B. L. S. (1969), Estimation of a unimodal density, Sankhya, Ser. A., 31, 23–36.
  • Rao, M. M. and Ren, Z. D. (1991), Theory of Orlicz spaces, CRC Press, New York.
  • Rockafellar, R. T. (1997), Convex Analysis, Princeton Univ. Press, Princeton, New Jersey.
  • Rufibach, K. (2007), Computing Maximum Likelihood Estimators of a log-concave Density Function, J. Stat. Comput. Simul., 77, 561–574.
  • Schuhmacher, D. and Dümbgen, L. (2010), Consistency of Multivariate Log-Concave Density Estimators, Statist. Probab. Lett., 80, 376–380.
  • Schuhmacher, D., Hüsler, A. and Dümbgen, L. (2009), Multivariate Log-Concave Distributions as a Nearly Parametric Model. Preprint., http://arxiv.org/pdf/0907.0250.
  • Seregin, A. and Wellner, J. A. (2009), Nonparametric estimation of multivariate convex-transformed densities. Preprint., http://uk.arxiv.org/pdf/0911.4151.pdf.
  • van de Geer, S. (2000), Empirical Processes in M-estimation, Cambridge Univ. Press, Cambridge.
  • van de Geer, S. (1993), Hellinger-consistency of certain nonparametric maximum likelihood estimators, Ann. Statist., 21, 14–44.
  • van der Vaart, A. W. and Wellner, J. A. (1996), Weak Convergence and Empirical Processes, Springer-Verlag, New York.
  • Walther, G. (2002), Detecting the presence of mixing with multiscale maximum likelihood, J. Amer. Statist. Assoc., 97, 508–513.
  • Walther, G. (2010), Inference and modeling with log-concave distributions, Statistical Science, to appear.