The Annals of Statistics

Quasi-concave density estimation

Roger Koenker and Ivan Mizera

Full-text: Open access

Abstract

Maximum likelihood estimation of a log-concave probability density is formulated as a convex optimization problem and shown to have an equivalent dual formulation as a constrained maximum Shannon entropy problem. Closely related maximum Renyi entropy estimators that impose weaker concavity restrictions on the fitted density are also considered, notably a minimum Hellinger discrepancy estimator that constrains the reciprocal of the square-root of the density to be concave. A limiting form of these estimators constrains solutions to the class of quasi-concave densities.

Article information

Source
Ann. Statist. Volume 38, Number 5 (2010), 2998-3027.

Dates
First available in Project Euclid: 20 August 2010

Permanent link to this document
http://projecteuclid.org/euclid.aos/1282315406

Digital Object Identifier
doi:10.1214/10-AOS814

Mathematical Reviews number (MathSciNet)
MR2722462

Zentralblatt MATH identifier
1200.62031

Subjects
Primary: 62G07: Density estimation 62H12: Estimation
Secondary: 62G05: Estimation 62B10: Information-theoretic topics [See also 94A17] 90C25: Convex programming 94A17: Measures of information, entropy

Keywords
Density estimation unimodal strongly unimodal shape constraints convex optimization duality entropy semidefinite programming

Citation

Koenker, Roger; Mizera, Ivan. Quasi-concave density estimation. Ann. Statist. 38 (2010), no. 5, 2998--3027. doi:10.1214/10-AOS814. http://projecteuclid.org/euclid.aos/1282315406.


Export citation

References

  • Andersen, E. D. (2006). The MOSEK optimization tools manual, Version 4.0. Available at http://www.mosek.com.
  • Avriel, M. (1972). r-convex functions. Math. Program. 2 309–323.
  • Balabdaoui, F., Rufibach, K. and Wellner, J. A. (2009). Limit distribution theory for maximum likelihood estimation of a log-concave density. Ann. Statist. 37 1299–1331.
  • Basu, A., Harris, I. R., Hjort, N. L. and Jones, M. C. (1998). Robust and efficient estimation by minimising a density power divergence. Biometrika 85 549–559.
  • Birgé, L. and Massart, P. (1993). Rates of convergence for minimum contrast estimators. Probab. Theory Related Fields 97 113–150.
  • Borell, C. (1975). Convex set functions in d-space. Period. Math. Hungar. 6 111–136.
  • Borwein, J. and Lewis, A. S. (2006). Convex Analysis and Nonlinear Optimization: Theory and Examples. Springer, New York.
  • Caplin, A. and Nalebuff, B. (1991). Aggregation and social choice: A mean voter theorem. Econometrica 59 1–23.
  • Chang, G. T. and Walther, G. (2007). Clustering with mixtures of log-concave distributions. Comput. Statist. Data Anal. 51 6242–6251.
  • Cule, M., Gramacy, R. and Samworth, R. (2009). LogConcDEAD: An R package for maximum likelihood estimation of a multivariate log-concave density. J. Statist. Software 29 1–20.
  • Cule, M., Samworth, R. and Stewart, M. (2010). Computing the maximum likelihood estimator of a multidimensional log-concave density. J. Roy. Statist. Soc. Ser. B. To appear.
  • Cule, M. and Samworth, R. (2010). Theoretical properties of the log-concave maximum likelihood estimator of a multidimensional density. Electron. J. Statist. 4 254–270.
  • de Finetti, B. (1949). Sulle stratificazioni convesse. Ann. Mat. Pura Appl. 30 173–183.
  • Dharmadhikari, S. and Joag-Dev, K. (1988). Unimodality, Convexity and Applications. Academic Press, Boston.
  • Dümbgen, L., Hüsler, A. and Rufibach, K. (2007). Active set and EM Algorithms for log-concave densities based on complete and censored data. Available at arXiv:0707.4643v2.
  • Dümbgen, L. and Rufibach, K. (2009). Maximum likelihood estimation of a log-concave density: Basic properties and uniform consistency. Bernoulli 15 40–68.
  • Good, I. J. (1971). A nonparametric roughness penalty for probability densities. Nature 229 29–30.
  • Grenander, U. (1956). On the theory of mortality measurement, part II. Skand. Aktuarietidskr. 39 125–153.
  • Groeneboom, P., Jongbloed, G. and Wellner, J. A. (2001). Estimation of a convex function: Characterizations and asymptotic theory. Ann. Statist. 29 1653–1698.
  • Gu, C. (2002). Smoothing Spline ANOVA Models. Springer, New York.
  • Hall, P. and Presnell, B. (1999). Density estimation under constraints. J. Comput. Graph. Statist. 8 259–277.
  • Hardy, G. H., Littlewood, J. E. and Pólya, G. (1934). Inequalities. Cambridge Univ. Press, London.
  • Hiriart-Urruty, J.-B. and Lemaréchal, C. (1993). Convex Analysis and Minimization Algorithms I, II. Springer, Berlin.
  • Hoffleit, D. and Warren, W. H. (1991). The Bright Star Catalog, 5th ed. Yale Univ. Observatory, New Haven, CT.
  • Huber, P. J. (1967). The behavior of maximum likelihood estimates under nonstandard conditions. In Proc. Fifth Berkeley Sympos. Math. Statist. Probability 1 221–233. Univ. California Press, Berkeley.
  • Karlin, S. (1968). Total Positivity. Stanford Univ. Press, Stanford, CA.
  • Koenker, R. and Mizera, I. (2008). Primal and dual formulations relevant for the numerical estimation of a probability density via regularization. Tatra Mt. Math. Publ. 39 255–264.
  • Kooperberg, C. and Stone, C. J. (1991). A study of logspline density estimation. Comput. Statist. Data Anal. 12 327–347.
  • MacDonell, W. R. (1902). On criminal anthropometry and the identification of criminals. Biometrika 1 177–227.
  • Mizera, I. (2002). On depth and deep points: A calculus. Ann. Statist. 30 1681–1736.
  • Müller, S. and Rufibach, K. (2009). Smooth tail-index estimation. J. Stat. Comput. Simul. 79 1155–1167.
  • Pal, J. K., Woodroofe, M. and Meyer, M. (2007). Estimating a Polya frequency function. In Complex Datasets and Inverse Problems: Tomography, Networks and Beyond (R. Liu, W. Strawderman and C.-H. Zhang, eds.). IMS Lecture Notes-Monograph Series 54 239–249. IMS, Beachwood, OH.
  • Prékopa, A. (1973). On logarithmic concave measures and functions. Acta Sci. Math. (Szeged) 34 335–343.
  • Rényi, A. (1961). On measures of entropy and information. In Proc. Fourth Berkeley Symp. Math. Statist. Probab. 547–561. Univ. California Press, Berkeley.
  • Rényi, A. (1965). On the foundations of information theory. Rev. d’Inst. Int. Statist. 33 1–14.
  • Rockafellar, R. T. (1966). Extensions of Fenchel’s duality theorem for convex functions. Duke Math. J. 33 81–89.
  • Rockafellar, R. T. (1970). Convex Analysis. Princeton Univ. Press, Princeton.
  • Rockafellar, R. T. (1971). Integrals which are convex functionals, II. Pacific J. Math. 39 439–469.
  • Rockafellar, R. T. (1974). Conjugate Duality and Optimization. SIAM, Philadelphia.
  • Rufibach, K. (2007). Computing maximum likelihood estimators of a log-concave density function. J. Stat. Comput. Simul. 77 561–574.
  • Rufibach, K. and Dümbgen, L. (2009). Logcondens: Estimate a log-concave probability density from iid observations. R package Version 1.3.4. Available at http://CRAN.R-project.org/package=logcondens.
  • Saunders, M. A. (2003). PDCO: A primal-dual interior solver for convex optimization. Available at http://www.stanford.edu/group/SOL/software/pdco.html.
  • Schennach, S. (2007). Estimation with exponentially tilted empirical likelihood. Ann. Statist. 35 634–672.
  • Seregin, A. and Wellner, J. A. (2009). Nonparametric estimation of multivariate convex-transformed densities. Available at arXiv:0911.4151v1.
  • Silverman, B. W. (1981). Using kernel density estimates to investigate multimodality. J. Roy. Statist. Soc. Ser. B 43 97–99.
  • Silverman, B. W. (1982). On the estimation of a probability density function by the maximum penalized likelihood method. Ann. Statist. 10 795–810.
  • Silverman, B. W. (1986). Density Estimation for Statistics and Data Analysis. Chapman and Hall, London.
  • Student (1908). The probable error of the mean. Biometrika 6 1–23.
  • Walther, G. (2001). Multiscale maximum likelihood analysis of a semiparametric model with applications. Ann. Statist. 29 1298–1319.
  • Walther, G. (2002). Detecting the presence of mixing with multiscale maximum likelihood. J. Amer. Statist. Assoc. 97 508–513.
  • Walther, G. (2009). Inference and modeling with log-concave distributions. Statist. Sci. 24 319–327.
  • Zeidler, E. (1985). Nonlinear Functional Analysis and Its Applications III: Variational Methods and Optimization. Springer, New York.