The Annals of Statistics

Global rates of convergence in log-concave density estimation

Abstract

The estimation of a log-concave density on $\mathbb{R}^{d}$ represents a central problem in the area of nonparametric inference under shape constraints. In this paper, we study the performance of log-concave density estimators with respect to global loss functions, and adopt a minimax approach. We first show that no statistical procedure based on a sample of size $n$ can estimate a log-concave density with respect to the squared Hellinger loss function with supremum risk smaller than order $n^{-4/5}$, when $d=1$, and order $n^{-2/(d+1)}$ when $d\geq2$. In particular, this reveals a sense in which, when $d\geq3$, log-concave density estimation is fundamentally more challenging than the estimation of a density with two bounded derivatives (a problem to which it has been compared). Second, we show that for $d\leq3$, the Hellinger $\varepsilon$-bracketing entropy of a class of log-concave densities with small mean and covariance matrix close to the identity grows like $\max\{\varepsilon^{-d/2},\varepsilon^{-(d-1)}\}$ (up to a logarithmic factor when $d=2$). This enables us to prove that when $d\leq3$ the log-concave maximum likelihood estimator achieves the minimax optimal rate (up to logarithmic factors when $d=2,3$) with respect to squared Hellinger loss.

Article information

Source
Ann. Statist., Volume 44, Number 6 (2016), 2756-2779.

Dates
Revised: March 2016
First available in Project Euclid: 23 November 2016

https://projecteuclid.org/euclid.aos/1479891634

Digital Object Identifier
doi:10.1214/16-AOS1480

Mathematical Reviews number (MathSciNet)
MR3576560

Zentralblatt MATH identifier
1360.62157

Subjects
Primary: 62G07: Density estimation 62G20: Asymptotic properties

Citation

Kim, Arlene K. H.; Samworth, Richard J. Global rates of convergence in log-concave density estimation. Ann. Statist. 44 (2016), no. 6, 2756--2779. doi:10.1214/16-AOS1480. https://projecteuclid.org/euclid.aos/1479891634

References

• Aleksandrov, A. D. (1939). Almost everywhere existence of the second differential of a convex functions and related properties of convex surfaces. Uchenye Zapisky Leningrad. Gos. Univ. Math. Ser. 37 3–35.
• Birgé, L. and Massart, P. (1993). Rates of convergence for minimum contrast estimators. Probab. Theory Related Fields 97 113–150.
• Brunel, V.-E. (2013). Adaptive estimation of convex polytopes and convex sets from noisy data. Electron. J. Stat. 7 1301–1327.
• Brunel, V.-E. (2016). Adaptive estimation of convex and polytopal density support. Probab. Theory Related Fields 164 1–16.
• Chen, Y. and Samworth, R. J. (2013). Smoothed log-concave maximum likelihood estimation with applications. Statist. Sinica 23 1373–1398.
• Cule, M. and Samworth, R. (2010). Theoretical properties of the log-concave maximum likelihood estimator of a multidimensional density. Electron. J. Stat. 4 254–270.
• Cule, M., Samworth, R. and Stewart, M. (2010). Maximum likelihood estimation of a multi-dimensional log-concave density. J. R. Stat. Soc. Ser. B. Stat. Methodol. 72 545–607.
• Doss, C. R. and Wellner, J. A. (2016). Global rates of convergence of the MLEs of log-concave and $s$-concave densities. Ann. Statist. 44 954–981.
• Dümbgen, L. and Rufibach, K. (2009). Maximum likelihood estimation of a log-concave density and its distribution function: Basic properties and uniform consistency. Bernoulli 15 40–68.
• Dümbgen, L., Samworth, R. and Schuhmacher, D. (2011). Approximation by log-concave distributions, with applications to regression. Ann. Statist. 39 702–730.
• Fresen, D. (2013). A multivariate Gnedenko law of large numbers. Ann. Probab. 41 3051–3080.
• Gao, F. and Wellner, J. A. (2015). Entropy of convex functions on $\mathbb{R}^{d}$. Available at http://arxiv.org/abs/1502.01752.
• Gerschgorin, S. (1931). Über die Abgrenzung der Eigenwerte einer Matrix. Izv. Akad. Nauk. USSR Otd. Fiz.-Mat. Nauk 6 749–754.
• Gradshteyn, I. S. and Ryzhik, I. M. (2007). Table of Integrals, Series, and Products, 7th ed. Elsevier/Academic Press, Amsterdam.
• Guntuboyina, A. and Sen, B. (2013). Covering numbers for convex functions. IEEE Trans. Inform. Theory 59 1957–1965.
• Ibragimov, I. A. and Khas’minskii, R. Z. (1983). Estimation of distribution density. J. Sov. Math. 25 40–57.
• Kim, A. K. H. and Samworth, R. J. (2015). Global rates of convergence in log-concave density estimation. Available at http://arxiv.org/abs/1404.2298v2.
• Kim, A. K. H. and Samworth, R. J. (2016). Supplement to “Global rates of convergence in log-concave density estimation.” DOI:10.1214/16-AOS1480SUPP.
• Korostelëv, A. P. and Tsybakov, A. B. (1993). Minimax Theory of Image Reconstruction. Lecture Notes in Statistics 82. Springer, New York.
• Lovász, L. and Vempala, S. (2007). The geometry of logconcave functions and sampling algorithms. Random Structures Algorithms 30 307–358.
• Mammen, E. and Tsybakov, A. B. (1995). Asymptotical minimax recovery of sets with smooth boundaries. Ann. Statist. 23 502–524.
• Müller, S. and Rufibach, K. (2009). Smooth tail-index estimation. J. Stat. Comput. Simul. 79 1155–1167.
• Pal, J. K., Woodroofe, M. and Meyer, M. (2007). Complex Datasets and Inverse Problems. Institute of Mathematical Statistics Lecture Notes—Monograph Series 54 239–249. IMS, Beachwood, OH.
• Samworth, R. J. and Yuan, M. (2012). Independent component analysis via nonparametric maximum likelihood estimation. Ann. Statist. 40 2973–3002.
• Schuhmacher, D. and Dümbgen, L. (2010). Consistency of multivariate log-concave density estimators. Statist. Probab. Lett. 80 376–380.
• Seregin, A. and Wellner, J. A. (2010). Nonparametric estimation of multivariate convex-transformed densities. Ann. Statist. 38 3751–3781.
• van de Geer, S. (2000). Empirical Processes in $M$-Estimation. Cambridge Univ. Press, Cambridge.
• van der Vaart, A. W. and Wellner, J. A. (1996). Weak Convergence and Empirical Processes. Springer, New York.
• Walther, G. (2002). Detecting the presence of mixing with multiscale maximum likelihood. J. Amer. Statist. Assoc. 97 508–513.
• Yang, Y. and Barron, A. (1999). Information-theoretic determination of minimax rates of convergence. Ann. Statist. 27 1564–1599.

Supplemental materials

• Supplementary material to “Global rates of convergence in log-concave density estimation”. Proof of Theorem 1 and auxiliary results.