The Annals of Statistics

Adaptation to lowest density regions with application to support recovery

Tim Patschkowski and Angelika Rohde

Full-text: Open access


A scheme for locally adaptive bandwidth selection is proposed which sensitively shrinks the bandwidth of a kernel estimator at lowest density regions such as the support boundary which are unknown to the statistician. In case of a Hölder continuous density, this locally minimax-optimal bandwidth is shown to be smaller than the usual rate, even in case of homogeneous smoothness. Some new type of risk bound with respect to a density-dependent standardized loss of this estimator is established. This bound is fully nonasymptotic and allows to deduce convergence rates at lowest density regions that can be substantially faster than $n^{-1/2}$. It is complemented by a weighted minimax lower bound which splits into two regimes depending on the value of the density. The new estimator adapts into the second regime, and it is shown that simultaneous adaptation into the fastest regime is not possible in principle as long as the Hölder exponent is unknown. Consequences on plug-in rules for support recovery are worked out in detail. In contrast to those with classical density estimators, the plug-in rules based on the new construction are minimax-optimal, up to some logarithmic factor.

Article information

Ann. Statist., Volume 44, Number 1 (2016), 255-287.

Received: September 2014
Revised: May 2015
First available in Project Euclid: 10 December 2015

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G07: Density estimation

Anisotropic density estimation bandwidth selection adaptation to lowest density regions density dependent minimax optimality support estimation


Patschkowski, Tim; Rohde, Angelika. Adaptation to lowest density regions with application to support recovery. Ann. Statist. 44 (2016), no. 1, 255--287. doi:10.1214/15-AOS1366.

Export citation


  • Baíllo, A., Cuevas, A. and Justel, A. (2000). Set estimation and nonparametric detection. Canad. J. Statist. 28 765–782.
  • Bertin, K., Lacour, C. and Rivoirard, V. (2014). Adaptive pointwise estimation of conditional density function. Available at arXiv:1312.7402.
  • Bhattacharya, R. N. and Ranga Rao, R. (1976). Normal Approximation and Asymptotic Expansions. Wiley, New York.
  • Biau, G., Cadre, B. and Pelletier, B. (2008). Exact rates in density support estimation. J. Multivariate Anal. 99 2185–2207.
  • Biau, G., Cadre, B., Mason, D. M. and Pelletier, B. (2009). Asymptotic normality in density support estimation. Electron. J. Probab. 14 2617–2635.
  • Birgé, L. (2014). Model selection for density estimation with $\mathbb{L}_{2}$-loss. Probab. Theory Related Fields 158 533–574.
  • Brunel, V.-E. (2013). Adaptive estimation of convex and polytopal density support. Probab. Theory Related Fields. To appear. Available at arXiv:1309.6602.
  • Butucea, C. (2001). Exact adaptive pointwise estimation on Sobolev classes of densities. ESAIM Probab. Stat. 5 1–31 (electronic).
  • Cai, T. T., Low, M. G. and Zhao, L. H. (2007). Trade-offs between global and local risks in nonparametric function estimation. Bernoulli 13 1–19.
  • Chavel, I. (2001). Isoperimetric Inequalities: Differential Geometric and Analytic Perspectives. Cambridge Tracts in Mathematics 145. Cambridge Univ. Press, Cambridge.
  • Chevalier, J. (1976). Estimation du support et du contour du support d’une loi de probabilité. Ann. Inst. H. Poincaré Sect. B (N.S.) 12 339–364.
  • Chichignoud, M. (2012). Minimax and minimax adaptive estimation in multiplicative regression: Locally Bayesian approach. Probab. Theory Related Fields 153 543–586.
  • Chichignoud, M. and Lederer, J. (2014). A robust, adaptive M-estimator for pointwise estimation in heteroscedastic regression. Bernoulli 20 1560–1599.
  • Cholaquidis, A., Cuevas, A. and Fraiman, R. (2014). On Poincaré cone property. Ann. Statist. 42 255–284.
  • Cuevas, A. (1990). On pattern analysis in the nonconvex case. Kybernetes 19 26–33.
  • Cuevas, A. and Fraiman, R. (1997). A plug-in approach to support estimation. Ann. Statist. 25 2300–2312.
  • Cuevas, A. and Rodríguez-Casal, A. (2004). On boundary estimation. Adv. in Appl. Probab. 36 340–354.
  • Dattner, I., Reiss, M. and Trabs, M. (2014). Adaptive quantile estimation in deconvolution with unknown error distribution. Bernoulli. To appear.
  • Devroye, L. and Wise, G. L. (1980). Detection of abnormal behavior via nonparametric estimation of the support. SIAM J. Appl. Math. 38 480–488.
  • Efromovich, S. (2008). Adaptive estimation of and oracle inequalities for probability densities and characteristic functions. Ann. Statist. 36 1127–1155.
  • Gach, F., Nickl, R. and Spokoiny, V. (2013). Spatially adaptive density estimation by localised Haar projections. Ann. Inst. Henri Poincaré Probab. Stat. 49 900–914.
  • Gayraud, G. (1997). Estimation of functionals of density support. Math. Methods Statist. 6 26–46.
  • Geffroy, J. (1964). Sur un problème d’estimation géométrique. Publ. Inst. Statist. Univ. Paris 13 191–210.
  • Giné, E. and Nickl, R. (2010). Confidence bands in density estimation. Ann. Statist. 38 1122–1170.
  • Goldenshluger, A. and Lepski, O. (2011). Bandwidth selection in kernel density estimation: Oracle inequalities and adaptive minimax optimality. Ann. Statist. 39 1608–1632.
  • Goldenshluger, A. and Lepski, O. (2014). On adaptive minimax density estimation on $R^{d}$. Probab. Theory Related Fields 159 479–543.
  • Grenander, U. (1981). Abstract Inference. Wiley, New York.
  • Groeneboom, P. (1988). Limit theorems for convex hulls. Probab. Theory Related Fields 79 327–368.
  • Hall, P. (1982). On estimating the endpoint of a distribution. Ann. Statist. 10 556–568.
  • Hall, P., Nussbaum, M. and Stern, S. E. (1997). On the estimation of a support curve of indeterminate sharpness. J. Multivariate Anal. 62 204–232.
  • Härdle, W., Park, B. U. and Tsybakov, A. B. (1995). Estimation of non-sharp support boundaries. J. Multivariate Anal. 55 205–218.
  • Jirak, M., Meister, A. and Reiss, M. (2014). Adaptive function estimation in nonparametric regression with one-sided errors. Ann. Statist. 42 1970–2002.
  • Juditsky, A. and Lambert-Lacroix, S. (2004). On minimax density estimation on $\mathbb{R}$. Bernoulli 10 187–220.
  • Kerkyacharian, G., Lepski, O. and Picard, D. (2001). Nonlinear estimation in anisotropic multi-index denoising. Probab. Theory Related Fields 121 137–170.
  • Klemelä, J. (2004). Complexity penalized support estimation. J. Multivariate Anal. 88 274–297.
  • Klutchnikoff, N. (2005). Sur l’estimation adaptative de fonctions anisotropes. Ph.D. Thesis, Univ. Aix-Marseille I.
  • Korostelëv, A. P. and Tsybakov, A. B. (1993). Minimax Theory of Image Reconstruction. Lecture Notes in Statistics 82. Springer, New York.
  • Lepski, O. V. (1990). A problem of adaptive estimation in Gaussian white noise. Theory Probab. Appl. 35 459–470.
  • Lepski, O. (2013). Multivariate density estimation under sup-norm loss: Oracle approach, adaptation and independence structure. Ann. Statist. 41 1005–1034.
  • Lepski, O. (2015). Adaptive estimation over anisotropic functional classes via oracle approach. Ann. Statist. 43 1178–1242.
  • Liu, L. and Wong, W. H. (2014). Multivariate density estimation based on adaptive partitioning: Convergence rate, variable selection and spatial adaptation. Available at arXiv:1401.2597.
  • Mammen, E. and Tsybakov, A. B. (1995). Asymptotical minimax recovery of sets with smooth boundaries. Ann. Statist. 23 502–524.
  • Mammen, E. and Tsybakov, A. B. (1999). Smooth discrimination analysis. Ann. Statist. 27 1808–1829.
  • Nussbaum, M. (1996). Asymptotic equivalence of density estimation and Gaussian white noise. Ann. Statist. 24 2399–2430.
  • Patschkowski, T. and Rohde, A. (2015). Supplement to “Adaptation to lowest density regions with application to support recovery.” DOI:10.1214/15-AOS1366SUPP.
  • Polonik, W. (1995). Measuring mass concentrations and estimating density contour clusters—An excess mass approach. Ann. Statist. 23 855–881.
  • Rényi, A. and Sulanke, R. (1963). Über die konvexe Hülle von $n$ zufällig gewählten Punkten. Z. Wahrsch. Verw. Gebiete 2 75–84.
  • Rényi, A. and Sulanke, R. (1964). Über die konvexe Hülle von $n$ zufällig gewählten Punkten. II. Z. Wahrsch. Verw. Gebiete 3 138–147.
  • Reynaud-Bouret, P., Rivoirard, V. and Tuleau-Malot, C. (2011). Adaptive density estimation: A curse of support? J. Statist. Plann. Inference 141 115–139.
  • Rigollet, P. and Vert, R. (2009). Optimal rates for plug-in estimators of density level sets. Bernoulli 15 1154–1178.
  • Rohde, A. (2008). Adaptive goodness-of-fit tests based on signed ranks. Ann. Statist. 36 1346–1374.
  • Rohde, A. (2011). Optimal calibration for multiple testing against local inhomogeneity in higher dimension. Probab. Theory Related Fields 149 515–559.
  • Tsybakov, A. B. (1989). Optimal estimation accuracy of nonsmooth images. Problems of Information Transmission 25 180–191.
  • Tsybakov, A. B. (1991). Nonparametric techniques in image estimation. In Nonparametric Functional Estimation and Related Topics (Spetses, 1990) (G. Roussas, ed.). NATO Adv. Sci. Inst. Ser. C Math. Phys. Sci. 335 669–677. Kluwer Academic, Dordrecht.
  • Tsybakov, A. B. (1997). On nonparametric estimation of density level sets. Ann. Statist. 25 948–969.
  • Tsybakov, A. B. (2004). Optimal aggregation of classifiers in statistical learning. Ann. Statist. 32 135–166.

Supplemental materials

  • Supplement to “Adaptation to lowest density regions with application to support recovery”. Supplement A is organized as follows. Section A.1 contains the proofs of Lemmas 5.1–5.6, which are central ingredients for the proof of Theorem 3.3. Section A.2 is concerned with the remaining proofs of Section 3. Section A.3 contains the proofs of Section 4. Section A.4 introduces a specific construction of a kernel function with prescribed Hölder regularity, which is frequently used throughout the article.