The Annals of Statistics

Bandwidth selection in kernel density estimation: Oracle inequalities and adaptive minimax optimality

Alexander Goldenshluger and Oleg Lepski

Full-text: Open access


We address the problem of density estimation with $\mathbb{L}_{s}$-loss by selection of kernel estimators. We develop a selection procedure and derive corresponding $\mathbb{L}_{s}$-risk oracle inequalities. It is shown that the proposed selection rule leads to the estimator being minimax adaptive over a scale of the anisotropic Nikol’skii classes. The main technical tools used in our derivations are uniform bounds on the $\mathbb{L}_{s}$-norms of empirical processes developed recently by Goldenshluger and Lepski [Ann. Probab. (2011), to appear].

Article information

Ann. Statist. Volume 39, Number 3 (2011), 1608-1632.

First available in Project Euclid: 7 June 2011

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G05: Estimation 62G20: Asymptotic properties

Density estimation kernel estimators Ls-risk oracle inequalities adaptive estimation empirical process


Goldenshluger, Alexander; Lepski, Oleg. Bandwidth selection in kernel density estimation: Oracle inequalities and adaptive minimax optimality. Ann. Statist. 39 (2011), no. 3, 1608--1632. doi:10.1214/11-AOS883.

Export citation


  • Birgé, L. (2008). Model selection for density estimation with L2-loss. Available at arXiv:0808.1416v2.
  • Bretagnolle, J. and Huber, C. (1979). Estimation des densités: Risque minimax. Z. Wahrsch. Verw. Gebiete 47 119–137.
  • Devroye, L. and Györfi, L. (1985). Nonparametric Density Estimation: The L1 View. Wiley, New York.
  • Devroye, L. and Lugosi, G. (1996). A universally acceptable smoothing factor for kernel density estimates. Ann. Statist. 24 2499–2512.
  • Devroye, L. and Lugosi, G. (1997). Nonasymptotic universal smoothing factors, kernel complexity and Yatracos classes. Ann. Statist. 25 2626–2637.
  • Devroye, L. and Lugosi, G. (2001). Combinatorial Methods in Density Estimation. Springer, New York.
  • Donoho, D. L., Johnstone, I. M., Kerkyacharian, G. and Picard, D. (1996). Density estimation by wavelet thresholding. Ann. Statist. 24 508–539.
  • Goldenshluger, A. and Lepski, O. (2008). Universal pointwise selection rule in multivariate function estimation. Bernoulli 14 1150–1190.
  • Goldenshluger, A. and Lepski, O. (2009). Structural adaptation via Lp-norm oracle inequalities. Probab. Theory Related Fields 143 41–71.
  • Goldenshluger, A. and Lepski, O. (2011). Uniform bounds for norms of sums of independent random functions. Ann. Probab. To appear. Available at arXiv:0904.1950v2.
  • Hasminskii, R. and Ibragimov, I. (1990). On density estimation in the view of Kolmogorov’s ideas in approximation theory. Ann. Statist. 18 999–1010.
  • Ibragimov, I. A. and Has’minskiĭ, R. Z. (1980). An estimate of the density of a distribution. Zap. Nauchn. Sem. Leningrad. Otdel. Mat. Inst. Steklov. (LOMI) 98 61–85.
  • Ibragimov, I. A. and Khas’minskiĭ, R. Z. (1981). More on estimation of the density of a distribution. Zap. Nauchn. Sem. Leningrad. Otdel. Mat. Inst. Steklov. (LOMI) 108 72–88.
  • Jennrich, R. I. (1969). Asymptotic properties of non-linear least squares estimators. Ann. Math. Statist. 40 633–643.
  • Johnson, W. B., Schechtman, G. and Zinn, J. (1985). Best constants in moment inequalities for linear combinations of independent and exchangeable random variables. Ann. Probab. 13 234–253.
  • Juditsky, A. and Lambert-Lacroix, S. (2004). On minimax density estimation on ℝ. Bernoulli 10 187–220.
  • Kerkyacharian, G., Lepski, O. and Picard, D. (2001). Nonlinear estimation in anisotropic multi-index denoising. Probab. Theory Related Fields 121 137–170.
  • Kerkyacharian, G., Picard, D. and Tribouley, K. (1996). Lp adaptive density estimation. Bernoulli 2 229–247.
  • Mason, D. M. (2009). Risk bounds for kernel density estimators. Zap. Nauchn. Sem. Leningrad. Otdel. Mat. Inst. Steklov. (LOMI) 363 66–104. Available at
  • Massart, P. (2007). Concentration Inequalities and Model Selection. Lecture Notes in Math. 1896. Springer, Berlin.
  • Nikol’skiĭ, S. M. (1969). Priblizhenie Funktsii Mnogikh Peremennykh i Teoremy Vlozheniya. Nauka, Moscow.
  • Parzen, E. (1962). On estimation of a probability density function and mode. Ann. Math. Statist. 33 1065–1076.
  • Rigollet, P. and Tsybakov, A. B. (2007). Linear and convex aggregation of density estimators. Math. Methods Statist. 16 260–280.
  • Rosenblatt, M. (1956). Remarks on some nonparametric estimates of a density function. Ann. Math. Statist. 27 832–837.
  • Samarov, A. and Tsybakov, A. (2007). Aggregation of density estimators and dimension reduction. In Advances in Statistical Modeling and Inference (V. Nair, ed.). Ser. Biostat. 3 233–251. World Scientific, Hackensack, NJ.
  • Silverman, B. W. (1986). Density Estimation for Statistics and Data Analysis. Chapman & Hall, London.