The Annals of Statistics

Multivariate density estimation under sup-norm loss: Oracle approach, adaptation and independence structure

Oleg Lepski

Full-text: Open access


This paper deals with the density estimation on $\mathbb{R}^{d}$ under sup-norm loss. We provide a fully data-driven estimation procedure and establish for it a so-called sup-norm oracle inequality. The proposed estimator allows us to take into account not only approximation properties of the underlying density, but eventual independence structure as well. Our results contain, as a particular case, the complete solution of the bandwidth selection problem in the multivariate density model. Usefulness of the developed approach is illustrated by application to adaptive estimation over anisotropic Nikolskii classes.

Article information

Ann. Statist., Volume 41, Number 2 (2013), 1005-1034.

First available in Project Euclid: 29 May 2013

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G07: Density estimation

Density estimation oracle inequality adaptation upper function


Lepski, Oleg. Multivariate density estimation under sup-norm loss: Oracle approach, adaptation and independence structure. Ann. Statist. 41 (2013), no. 2, 1005--1034. doi:10.1214/13-AOS1109.

Export citation


  • Akakpo, N. (2012). Adaptation to anisotropy and inhomogeneity via dyadic piecewise polynomial selection. Math. Methods Statist. 21 1–28.
  • Bertin, K. (2005). Sharp adaptive estimation in sup-norm for $d$-dimensional Hölder classes. Math. Methods Statist. 14 267–298.
  • Birgé, L. (2008). Model selection for density estimation with $\mathbb{L}_{2}$-loss. Available at arXiv:0808.1416v2.
  • Bretagnolle, J. and Huber, C. (1979). Estimation des densités: Risque minimax. Z. Wahrsch. Verw. Gebiete 47 119–137.
  • Devroye, L. and Györfi, L. (1985). Nonparametric Density Estimation: The $\mathbb{L}_{1}$ View. Wiley, New York.
  • Devroye, L. and Lugosi, G. (1996). A universally acceptable smoothing factor for kernel density estimates. Ann. Statist. 24 2499–2512.
  • Devroye, L. and Lugosi, G. (1997). Nonasymptotic universal smoothing factors, kernel complexity and Yatracos classes. Ann. Statist. 25 2626–2637.
  • Devroye, L. and Lugosi, G. (2001). Combinatorial Methods in Density Estimation. Springer, New York.
  • Donoho, D. L., Johnstone, I. M., Kerkyacharian, G. and Picard, D. (1996). Density estimation by wavelet thresholding. Ann. Statist. 24 508–539.
  • Efroimovich, S. Y. (1986). Non-parametric estimation of the density with unknown smoothness. Theory Probab. Appl. 30 557–568.
  • Efromovich, S. Y. (2008). Adaptive estimation of and oracle inequalities for probability densities and characteristic functions. Ann. Statist. 36 1127–1155.
  • Gach, D., Nickl, R. and Spokoiny, V. (2013). Spatially adaptive density estimation by localised Haar projections. Ann. Inst. Henri Poincaré Probab. Stat. To appear.
  • Giné, E. and Nickl, R. (2009). An exponential inequality for the distribution function of the kernel density estimator, with applications to adaptive estimation. Probab. Theory Related Fields 143 569–596.
  • Giné, E. and Nickl, R. (2010). Adaptive estimation of a distribution function and its density in sup-norm loss by wavelet and spline projections. Bernoulli 16 1137–1163.
  • Goldenshluger, A. and Lepski, O. (2008). Universal pointwise selection rule in multivariate function estimation. Bernoulli 14 1150–1190.
  • Goldenshluger, A. and Lepski, O. (2009). Structural adaptation via $\mathbb{L}_{p}$-norm oracle inequalities. Probab. Theory Related Fields 143 41–71.
  • Goldenshluger, A. and Lepski, O. (2011). Bandwidth selection in kernel density estimation: Oracle inequalities and adaptive minimax optimality. Ann. Statist. 39 1608–1632.
  • Goldenshluger, A. and Lepski, O. (2012). On adaptive minimax density estimation on $\mathbb{R}^{d}$. Available at arXiv:1210.1715v1.
  • Golubev, G. K. (1992). Nonparametric estimation of smooth densities of a distribution in $L_{2}$. Probl. Peredachi Inf. 28 52–62.
  • Khasminskii, R. and Ibragimov, I. (1990). On density estimation in the view of Kolmogorov’s ideas in approximation theory. Ann. Statist. 18 999–1010.
  • Ibragimov, I. A. and Khasminskii, R. Z. (1980). An estimate of the density of a distribution. Zap. Nauchn. Sem. Leningrad. Otdel. Mat. Inst. Steklov. (LOMI) 98 61–85, 161–162, 166.
  • Ibragimov, I. A. and Khasminskii, R. Z. (1981). More on estimation of the density of a distribution. Zap. Nauchn. Sem. Leningrad. Otdel. Mat. Inst. Steklov. (LOMI) 108 72–88, 194, 198.
  • Jennrich, R. I. (1969). Asymptotic properties of non-linear least squares estimators. Ann. Math. Statist. 40 633–643.
  • Juditsky, A. and Lambert-Lacroix, S. (2004). On minimax density estimation on $\mathbb{R}$. Bernoulli 10 187–220.
  • Kerkyacharian, G., Lepski, O. and Picard, D. (2001). Nonlinear estimation in anisotropic multi-index denoising. Probab. Theory Related Fields 121 137–170.
  • Kerkyacharian, G., Lepski, O. and Picard, D. (2007). Nonlinear estimation in anisotropic multiindex denoising. Sparse case. Theory Probab. Appl. 52 150–171.
  • Kerkyacharian, G., Picard, D. and Tribouley, K. (1996). $L^{p}$ adaptive density estimation. Bernoulli 2 229–247.
  • Lepski, O. (2012). Upper functions for positive random functionals. Available at arXiv:1202.6615v1.
  • Lepski, O. V. and Levit, B. Y. (1999). Adaptive nonparametric estimation of smooth multivariate functions. Math. Methods Statist. 8 344–370.
  • Lepskiĭ, O. V. (1991). Asymptotically minimax adaptive estimation. I. Upper bounds. Optimally adaptive estimates. Theory Probab. Appl. 36 682–697.
  • Lepskiĭ, O. V. (1992). On problems of adaptive estimation in white Gaussian noise. In Topics in Nonparametric Estimation. Adv. Soviet Math. 12 87–106. Amer. Math. Soc., Providence, RI.
  • Mason, D. M. (2009). Risk bounds for kernel density estimators. Zap. Nauchn. Sem. S.-Peterburg. Otdel. Mat. Inst. Steklov. (POMI) 363 66–104, 183.
  • Massart, P. (2007). Concentration Inequalities and Model Selection. Lecture Notes in Math. 1896. Springer, Berlin.
  • Nikol’skiĭ, S. M. (1977). Priblizhenie Funktsii Mnogikh Peremennykh i Teoremy Vlozheniya, 2nd ed. “Nauka”, Moscow.
  • Parzen, E. (1962). On estimation of a probability density function and mode. Ann. Math. Statist. 33 1065–1076.
  • Reynaud-Bouret, P., Rivoirard, V. and Tuleau-Malot, C. (2011). Adaptive density estimation: A curse of support? J. Statist. Plann. Inference 141 115–139.
  • Rigollet, P. (2006). Adaptive density estimation using the blockwise Stein method. Bernoulli 12 351–370.
  • Rigollet, P. and Tsybakov, A. B. (2007). Linear and convex aggregation of density estimators. Math. Methods Statist. 16 260–280.
  • Rosenblatt, M. (1956). Remarks on some nonparametric estimates of a density function. Ann. Math. Statist. 27 832–837.
  • Samarov, A. and Tsybakov, A. (2007). Aggregation of density estimators and dimension reduction. In Advances in Statistical Modeling and Inference. Ser. Biostat. 3 233–251. World Sci. Publ., Hackensack, NJ.
  • Tsybakov, A. B. (1998). Pointwise and sup-norm sharp adaptive estimation of functions on the Sobolev classes. Ann. Statist. 26 2420–2469.