## The Annals of Statistics

### Approximation and estimation of s-concave densities via Rényi divergences

#### Abstract

In this paper, we study the approximation and estimation of $s$-concave densities via Rényi divergence. We first show that the approximation of a probability measure $Q$ by an $s$-concave density exists and is unique via the procedure of minimizing a divergence functional proposed by [Ann. Statist. 38 (2010) 2998–3027] if and only if $Q$ admits full-dimensional support and a first moment. We also show continuity of the divergence functional in $Q$: if $Q_{n}\to Q$ in the Wasserstein metric, then the projected densities converge in weighted $L_{1}$ metrics and uniformly on closed subsets of the continuity set of the limit. Moreover, directional derivatives of the projected densities also enjoy local uniform convergence. This contains both on-the-model and off-the-model situations, and entails strong consistency of the divergence estimator of an $s$-concave density under mild conditions. One interesting and important feature for the Rényi divergence estimator of an $s$-concave density is that the estimator is intrinsically related with the estimation of log-concave densities via maximum likelihood methods. In fact, we show that for $d=1$ at least, the Rényi divergence estimators for $s$-concave densities converge to the maximum likelihood estimator of a log-concave density as $s\nearrow0$. The Rényi divergence estimator shares similar characterizations as the MLE for log-concave distributions, which allows us to develop pointwise asymptotic distribution theory assuming that the underlying density is $s$-concave.

#### Article information

Source
Ann. Statist., Volume 44, Number 3 (2016), 1332-1359.

Dates
Revised: October 2015
First available in Project Euclid: 11 April 2016

https://projecteuclid.org/euclid.aos/1460381695

Digital Object Identifier
doi:10.1214/15-AOS1408

Mathematical Reviews number (MathSciNet)
MR3485962

Zentralblatt MATH identifier
1338.62105

Subjects
Primary: 62G07: Density estimation 62H12: Estimation
Secondary: 62G05: Estimation 62G20: Asymptotic properties

#### Citation

Han, Qiyang; Wellner, Jon A. Approximation and estimation of s -concave densities via Rényi divergences. Ann. Statist. 44 (2016), no. 3, 1332--1359. doi:10.1214/15-AOS1408. https://projecteuclid.org/euclid.aos/1460381695

#### References

• Avriel, M. (1972). $r$-convex functions. Math. Program. 2 309–323.
• Balabdaoui, F., Rufibach, K. and Wellner, J. A. (2009). Limit distribution theory for maximum likelihood estimation of a log-concave density. Ann. Statist. 37 1299–1331.
• Balabdaoui, F. and Wellner, J. A. (2007). Estimation of a $k$-monotone density: Limit distribution theory and the spline connection. Ann. Statist. 35 2536–2564.
• Basu, A., Harris, I. R., Hjort, N. L. and Jones, M. C. (1998). Robust and efficient estimation by minimising a density power divergence. Biometrika 85 549–559.
• Birgé, L. and Massart, P. (1993). Rates of convergence for minimum contrast estimators. Probab. Theory Related Fields 97 113–150.
• Borell, C. (1974). Convex measures on locally convex spaces. Ark. Mat. 12 239–252.
• Borell, C. (1975). Convex set functions in $d$-space. Period. Math. Hungar. 6 111–136.
• Brascamp, H. J. and Lieb, E. H. (1976). On extensions of the Brunn–Minkowski and Prékopa–Leindler theorems, including inequalities for log concave functions, and with an application to the diffusion equation. J. Funct. Anal. 22 366–389.
• Brunk, H. D. (1970). Estimation of isotonic regression. In Nonparametric Techniques in Statistical Inference (Proc. Sympos., Indiana Univ., Bloomington, Ind., 1969) 177–197. Cambridge Univ. Press, London.
• Cule, M. and Samworth, R. (2010). Theoretical properties of the log-concave maximum likelihood estimator of a multidimensional density. Electron. J. Stat. 4 254–270.
• Cule, M., Samworth, R. and Stewart, M. (2010). Maximum likelihood estimation of a multi-dimensional log-concave density. J. R. Stat. Soc. Ser. B Stat. Methodol. 72 545–607.
• Das Gupta, S. (1976). $S$-unimodal function: Related inequalities and statistical applications. Sankhyā Ser. B 38 301–314.
• Dharmadhikari, S. and Joag-Dev, K. (1988). Unimodality, Convexity, and Applications. Academic Press, Boston, MA.
• Doss, C. and Wellner, J. A. (2016). Global rates of convergence of the MLEs of log-concave and s-concave densities. Ann. Statist. 44 954–981.
• Dümbgen, L. and Rufibach, K. (2009). Maximum likelihood estimation of a log-concave density and its distribution function: Basic properties and uniform consistency. Bernoulli 15 40–68.
• Dümbgen, L., Samworth, R. and Schuhmacher, D. (2011). Approximation by log-concave distributions, with applications to regression. Ann. Statist. 39 702–730.
• Groeneboom, P. (1985). Estimating a monotone density. In Proceedings of the Berkeley Conference in Honor of Jerzy Neyman and Jack Kiefer, Vol. II (Berkeley, Calif., 1983). Wadsworth Statist./Probab. Ser. 539–555. Wadsworth, Belmont, CA.
• Groeneboom, P., Jongbloed, G. and Wellner, J. A. (2001). Estimation of a convex function: Characterizations and asymptotic theory. Ann. Statist. 29 1653–1698.
• Guntuboyina, A. and Sen, B. (2015). Global risk bounds and adaptation in univariate convex regression. Probab. Theory Related Fields 163 379–411.
• Han, Q. and Wellner, J. A. (2015). Supplement to “Approximation and estimation of $s$-concave densities via Rényi divergences.” DOI:10.1214/15-AOS1408SUPP.
• Jongbloed, G. (2000). Minimax lower bounds and moduli of continuity. Statist. Probab. Lett. 50 279–284.
• Kim, A. K. and Samworth, R. J. (2015). Global rates of convergence in log-concave density estimation. Preprint. Available at arXiv:1404.2298v2.
• Koenker, R. and Mizera, I. (2010). Quasi-concave density estimation. Ann. Statist. 38 2998–3027.
• Koenker, R. and Mizera, I. (2014). Convex optimization in R. J. Stat. Softw. 60(5) 1–23.
• Lang, R. (1986). A note on the measurability of convex sets. Arch. Math. (Basel) 47 90–92.
• MOSEK ApS Denmark (2011). The MOSEK Optimization Tools Manual, Version 6.0. Available at http://www.mosek.com.
• Pal, J. K., Woodroofe, M. and Meyer, M. (2007). Estimating a Polya frequency function${}_{2}$. In Complex Datasets and Inverse Problems. Institute of Mathematical Statistics Lecture Notes—Monograph Series 54 239–249. IMS, Beachwood, OH.
• Prakasa Rao, B. L. S. (1969). Estimation of a unimodal density. Sankhyā Ser. A 31 23–36.
• Rinott, Y. (1976). On convexity of measures. Ann. Probab. 4 1020–1026.
• Rockafellar, R. T. (1971). Integrals which are convex functionals. II. Pacific J. Math. 39 439–469.
• Rockafellar, R. T. (1997). Convex Analysis. Princeton Landmarks in Mathematics. Princeton Univ. Press, Princeton, NJ.
• Seregin, A. and Wellner, J. A. (2010). Nonparametric estimation of multivariate convex-transformed densities. Ann. Statist. 38 3751–3781.
• Uhrin, B. (1984). Some remarks about the convolution of unimodal functions. Ann. Probab. 12 640–645.
• van de Geer, S. A. (2000). Applications of Empirical Process Theory. Cambridge Series in Statistical and Probabilistic Mathematics 6. Cambridge Univ. Press, Cambridge.
• van der Vaart, A. W. and Wellner, J. A. (1996). Weak Convergence and Empirical Processes: With Applications to Statistics. Springer, New York.
• Walther, G. (2002). Detecting the presence of mixing with multiscale maximum likelihood. J. Amer. Statist. Assoc. 97 508–513.
• Wright, F. T. (1981). The asymptotic behavior of monotone regression estimates. Ann. Statist. 9 443–448.

#### Supplemental materials

• Supplement to “Approximation and estimation of $s$-concave densities via Rényi divergences”. In the supplement Han and Wellner (2015), we provide details of the omitted proofs for Sections 2, 3, 4 and 6 and some auxiliary results from convex analysis used in the main paper.