The Annals of Statistics

Optimal pointwise adaptive methods in nonparametric estimation

O. V. Lepski and V. G. Spokoiny

Full-text: Open access


The problem of optimal adaptive estimation of a function at a given point from noisy data is considered. Two procedures are proved to be asymptotically optimal for different settings.

First we study the problem of bandwidth selection for nonparametric pointwise kernel estimation with a given kernel. We propose a bandwidth selection procedure and prove its optimality in the asymptotic sense. Moreover, this optimality is stated not only among kernel estimators with a variable bandwidth. The resulting estimator is asymptotically optimal among all feasible estimators. The important feature of this procedure is that it is fully adaptive and it "works" for a very wide class of functions obeying a mild regularity restriction. With it the attainable accuracy of estimation depends on the function itself and is expressed in terms of the "ideal adaptive bandwidth" corresponding to this function and a given kernel.

The second procedure can be considered as a specialization of the first one under the qualitative assumption that the function to be estimated belongs to some Hölder class $\Sigma (\beta, L)$ with unknown parameters $\beta, L$. This assumption allows us to choose a family of kernels in an optimal way and the resulting procedure appears to be asymptotically optimal in the adaptive sense in any range of adaptation with $\beta \leq 2$.

Article information

Ann. Statist., Volume 25, Number 6 (1997), 2512-2546.

First available in Project Euclid: 30 August 2002

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G07: Density estimation
Secondary: 62G20: Asymptotic properties

Bandwidth selection Hölder-type constraints pointwise adaptive estimation


Lepski, O. V.; Spokoiny, V. G. Optimal pointwise adaptive methods in nonparametric estimation. Ann. Statist. 25 (1997), no. 6, 2512--2546. doi:10.1214/aos/1030741083.

Export citation


  • Bretagnolle, J. and Huber, C. (1979). Estimation des densites: risque minimax. Z. Wahrsch. Verw. Gebiete 47 119-137.
  • Brockmann, M., Gasser, T. and Herrmann, E. (1993). Locally adaptive bandwidth choice for kernel regression estimators. J. Amer. Statist. Assoc. 88 1302-1309.
  • Brown, L. D. and Low, M. G. (1996). Asy mptotic equivalence of nonparametric regression and white noise. Ann. Statist. 24 2384-2398.
  • Brown, L. D. and Low, M. G. (1992). Superefficiency and lack of adaptability in functional estimation. Technical report, Cornell Univ.
  • Donoho, D. L. and Johnstone, I. M. (1994). Ideal spatial adaptation by wavelet shrinkage. Biometrika 81 425-455.
  • Donoho, D. L. and Johnstone, I. M. (1992). Minimax estimation via wavelet shrinkage. Unpublished manuscript.
  • Donoho, D. L. and Johnstone, I. M. (1995). Adapting to unknown smoothness via wavelet shrinkage. J. Amer. Statist. Assoc. 90 1200-1224.
  • Donoho, D. L., Johnstone, I. M., Kerky acharian, G. and Picard, D. (1995). Wavelet shrinkage: asy mptopia (with discussion)? J. Roy al Statist. Soc. Ser. B 57 301-369.
  • Donoho, D. L. and Liu, R. C. (1991). Geometrizing rate of convergence. III. Ann. Statist. 19 668-701.
  • Donoho, D. L. and Low, M. G. (1992). Renormalization exponents and optimal pointwise rates of convergence. Ann. Statist. 20 944-970.
  • Efroimovich, S. Y. and Low, M. G. (1994). Adaptive estimates of linear functionals. Probab. Theory Related Fields 98 261-275.
  • Hall, P. and Johnstone, I. (1992). Empirical functionals and efficient smoothing parameter selection. J. Roy. Statist. Soc. Ser. B 54 475-530.
  • H¨ardle, W. and Marron, J. S. (1985). Optimal bandwidth selection in nonparametric regression function estimation. Ann. Statist. 13 1466-1481.
  • Ibragimov, I. A. and Khasminskii, R. Z. (1980). Estimates of signal, its derivatives, and point of maximum for Gaussian observations. Theory Probab. Appl. 25 703-716.
  • Ibragimov, I. A. and Khasminskii, R. Z. (1981). Statistical Estimation: Asy mptotic Theory. Springer, New York.
  • Jones, M. C., Marron, J. S. and Park, B. (1991). A simple root-n bandwidth selector. Ann. Statist. 19 1919-1932.
  • Juditsky, A. (1995). Adaptive wavelet estimators. Math. Methods Statist. To appear.
  • Kerky acharian, G. and Picard, D. (1993). Density estimation by kernel and wavelet method, optimality in Besov space. Statist. Probab. Lett. 18 327-336.
  • Korostelev, A. P. (1993). Exact asy mptotic minimax estimate for a nonparametric regression in the uniform norm. Theory Probab. Appl. 38 737-743.
  • Lepski, O. V. (1990). One problem of adaptive estimation in Gaussian white noise. Theory Probab. Appl. 35 459-470.
  • Lepski, O. V. (1991). Asy mptotic minimax adaptive estimation. 1. Upper bounds. Theory Probab. Appl. 36 645-659.
  • Lepski, O. V. (1992). Asy mptotic minimax adaptive estimation. 2. Statistical model without optimal adaptation. Adaptive estimators. Theory Probab. Appl. 37 468-481.
  • Lepski, O. V., Mammen, E. and Spokoiny, V. G. (1997). Ideal spatial adaptation to inhomogeneous smoothness: an approach based on kernel estimates with variable bandwidth selection. Ann. Statist. 25 929-947.
  • Low, M. G. (1992). Renormalizing upper and lower bounds for integrated risk in the white noise model. Ann. Statist. 20 577-589.
  • Marron, J. S. (1988). Automatic smoothing parameter selection: a survey. Empir. Econom. 13 187-208.
  • M ¨uller, H. G. and Stadtm ¨uller, U. (1987). Variable bandwidth kernel estimators of regression curves. Ann. Statist. 15 182-201.
  • Nemirovskii, A. (1985). On nonparametric estimation of smooth regression function. Soviet J. Comput. Sy stem Sci. 23 1-11.
  • Nussbaum, M. (1996). Asy mptotic equivalence of density estimation and white noise. Ann. Statist. 24 2399-2430.
  • Sacks, J. and Strawderman, W. (1982). Improvements of linear minimax estimates. In Statistical Decision Theory and Related Topics 3 (S. S. Gupta and J. O. Berger, eds.) 2 287-304. Academic Press, New York.
  • Staniswalis, J. G. S. (1989). Local bandwidth selection for kernel estimates. J. Amer. Statist. Assoc. 84 284-288.
  • Stone, C. J. (1982). Optimal global rates of convergence for nonparametric regression. Ann. Statist. 10 1040-1053.
  • Triebel, H. (1992). Theory of Function Spaces 2. Birkh¨auser, Basel.
  • Vieu, P. (1991). Nonparametric regression: optimal local bandwidth choice. J. Roy. Statist. Soc. Ser. B 53 453-464.