The Annals of Probability

Uniform limit theorems for wavelet density estimators

Evarist Giné and Richard Nickl

Full-text: Open access


Let pn(y)=∑kα̂kϕ(yk)+∑l=0jn−1kβ̂lk2l/2ψ(2lyk) be the linear wavelet density estimator, where ϕ, ψ are a father and a mother wavelet (with compact support), α̂k, β̂lk are the empirical wavelet coefficients based on an i.i.d. sample of random variables distributed according to a density p0 on ℝ, and jn∈ℤ, jn∞. Several uniform limit theorems are proved: First, the almost sure rate of convergence of sup y∈ℝ|pn(y)−Epn(y)| is obtained, and a law of the logarithm for a suitably scaled version of this quantity is established. This implies that sup y∈ℝ|pn(y)−p0(y)| attains the optimal almost sure rate of convergence for estimating p0, if jn is suitably chosen. Second, a uniform central limit theorem as well as strong invariance principles for the distribution function of pn, that is, for the stochastic processes $\sqrt{n}(F_{n}^{W}(s)-F(s))=\sqrt{n}\int_{-\infty}^{s}(p_{n}-p_{0})$, s∈ℝ, are proved; and more generally, uniform central limit theorems for the processes $\sqrt{n}\int(p_{n}-p_{0})f$, $f\in\mathcal{F}$, for other Donsker classes $\mathcal{F}$ of interest are considered. As a statistical application, it is shown that essentially the same limit theorems can be obtained for the hard thresholding wavelet estimator introduced by Donoho et al. [Ann. Statist. 24 (1996) 508–539].

Article information

Ann. Probab. Volume 37, Number 4 (2009), 1605-1646.

First available in Project Euclid: 21 July 2009

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G07: Density estimation
Secondary: 60F05: Central limit and other weak theorems 60F15: Strong theorems 60F17: Functional limit theorems; invariance principles

Wavelet density estimator sup-norm loss law of the logarithm rates of convergence uniform central limit theorem wavelet thresholding adaptive estimation


Giné, Evarist; Nickl, Richard. Uniform limit theorems for wavelet density estimators. Ann. Probab. 37 (2009), no. 4, 1605--1646. doi:10.1214/08-AOP447.

Export citation


  • Bickel, P. J. and Ritov, Y. (2003). Nonparametric estimators which can be “plugged-in.” Ann. Statist. 31 1033–1053.
  • de la Peña, V. H. and Giné, E. (1999). Decoupling: From Dependence to Independence, Randomly Stopped Processes. U-Statistics and Processes. Martingales and Beyond. Springer, New York.
  • Daubechies, I. (1992). Ten Lectures on Wavelets. CBMS-NSF Regional Conference Series in Applied Mathematics 61. SIAM, Philadelphia, PA.
  • Deheuvels, P. (2000). Uniform limit laws for kernel density estimators on possibly unbounded intervals. In Recent Advances in Reliability Theory (Bordeaux, 2000). 477–492. Birkhäuser, Boston.
  • Delyon, B. and Juditsky, A. (1996). On minimax wavelet estimators. Appl. Comput. Harmon. Anal. 3 215–228.
  • Donoho, D. L., Johnstone, I. M., Kerkyacharian, G. and Picard, D. (1995). Wavelet shrinkage: Asymptopia? J. Roy. Statist. Soc. Ser. B 57 301–369.
  • Donoho, D. L., Johnstone, I. M., Kerkyacharian, G. and Picard, D. (1996). Density estimation by wavelet thresholding. Ann. Statist. 24 508–539.
  • Donoho, D. L., Johnstone, I. M., Kerkyacharian, G. and Picard, D. (1997). Universal near minimaxity of wavelet shrinkage. In Festschrift for Lucien Le Cam 183–218. Springer, New York.
  • Doukhan, P. and León, J. R. (1990). Déviation quadratique d’estimateurs de densité par projections orthogonales. C. R. Acad. Sci. Paris Sér. I Math. 310 425–430.
  • Dudley, R. M. (1992). Fréchet differentiability, p-variation and uniform Donsker classes. Ann. Probab. 20 1968–1982.
  • Dudley, R. M. (1999). Uniform Central Limit Theorems. Cambridge Studies in Advanced Mathematics 63. Cambridge Univ. Press, Cambridge.
  • Dvoretzky, A., Kiefer, J. and Wolfowitz, J. (1956). Asymptotic minimax character of the sample distribution function and of the classical multinomial estimator. Ann. Math. Statist. 27 642–669.
  • Einmahl, U. and Mason, D. M. (1996). Some universal results on the behavior of increments of partial sums. Ann. Probab. 24 1388–1407.
  • Einmahl, U. and Mason, D. M. (2000). An empirical process approach to the uniform consistency of kernel-type function estimators. J. Theoret. Probab. 13 1–37.
  • Einmahl, U. and Mason, D. M. (2005). Uniform in bandwidth consistency of kernel-type function estimators. Ann. Statist. 33 1380–1403.
  • Giné, E. and Guillou, A. (2001). On consistency of kernel density estimators for randomly censored data: Rates holding uniformly over adaptive intervals. Ann. Inst. H. Poincaré Probab. Statist. 37 503–522.
  • Giné, E. and Guillou, A. (2002). Rates of strong uniform consistency for multivariate kernel density estimators. Ann. Inst. H. Poincaré Probab. Statist. 38 907–921.
  • Giné, E., Latała, R. and Zinn, J. (2000). Exponential and moment inequalities for U-statistics. In High Dimensional Probability, II (Seattle, WA, 1999). Progress in Probability 47 (E. Giné, D. M. Mason and J. A. Wellner, eds.) 13–38. Birkhäuser, Boston.
  • Giné, E. and Nickl, R. (2008). Uniform central limit theorems for kernel density estimators. Probab. Theory Related Fields 141 333–387.
  • Giné, E. and Nickl, R. (2009). An exponential inequality for the distribution function of the kernel density estimator, with applications to adaptive estimation. Probab. Theory Related Fields 143 569–596.
  • Golubev, Y., Lepski, O. and Levit, B. (2001). On adaptive estimation for the sup-norm losses. Math. Methods Statist. 10 23–37.
  • Hall, P., Kerkyacharian, G. and Picard, D. (1998). Block threshold rules for curve estimation using kernel and wavelet methods. Ann. Statist. 26 922–942.
  • Härdle, W., Kerkyacharian, G., Picard, D. and Tsybakov, A. (1998). Wavelets, Approximation, and Statistical Applications. Lecture Notes in Statistics 129. Springer, New York.
  • Juditsky, A. and Lambert-Lacroix, S. (2004). On minimax density estimation on ℝ. Bernoulli 10 187–220.
  • Kerkyacharian, G. and Picard, D. (1992). Density estimation in Besov spaces. Statist. Probab. Lett. 13 15–24.
  • Kerkyacharian, G., Picard, D. and Tribouley, K. (1996). Lp adaptive density estimation. Bernoulli 2 229–247.
  • Komlós, J., Major, P. and Tusnády, G. (1975). An approximation of partial sums of independent rv’s and the sample df. I. Z. Wahrsch. Verw. Gebiete 32 111–131.
  • Ledoux, M. (2001). The Concentration of Measure Phenomenon. Mathematical Surveys and Monographs 89. Amer. Math. Soc., Providence, RI.
  • Love, E. R. and Young, L. C. (1937). Sur une classe de fonctionelles linéaires. Fund. Math. 28 243–257.
  • Massiani, A. (2003). Vitesse de convergence uniforme presque sûre de l’estimateur linéaire par méthode d’ondelettes. C. R. Math. Acad. Sci. Paris 337 67–70.
  • Meyer, Y. (1992). Wavelets and Operators. Cambridge Studies in Advanced Mathematics 37. Cambridge Univ. Press, Cambridge.
  • Nickl, R. (2006). Empirical and Gaussian processes on Besov classes. In High Dimensional Probability. Institute of Mathematical Statistics Lecture Notes—Monograph Series 51 (E. Giné, V. Koltchinskii, W. Li and J. Zinn, eds.) 185–195. IMS, Beachwood, OH.
  • Nickl, R. (2007). Donsker-type theorems for nonparametric maximum likelihood estimators. Probab. Theory Related Fields 138 411–449.
  • Nickl, R. and Pötscher, B. M. (2007). Bracketing metric entropy rates and empirical central limit theorems for function classes of Besov- and Sobolev-type. J. Theoret. Probab. 20 177–199.
  • Nolan, D. and Pollard, D. (1987). U-processes: Rates of convergence. Ann. Statist. 15 780–799.
  • Stute, W. (1982). A law of the logarithm for kernel density estimators. Ann. Probab. 10 414–422.
  • Talagrand, M. (1994). Sharper bounds for Gaussian and empirical processes. Ann. Probab. 22 28–76.
  • Talagrand, M. (1996). New concentration inequalities in product spaces. Invent. Math. 126 505–563.
  • Triebel, H. (1983). Theory of Function Spaces. Birkhäuser, Basel.
  • Tsybakov, A. B. (1998). Pointwise and sup-norm sharp adaptive estimation of functions on the Sobolev classes. Ann. Statist. 26 2420–2469.
  • van der Vaart, A. W. and Wellner, J. A. (1996). Weak Convergence and Empirical Processes. Springer Series in Statistics. Springer, New York.
  • Vidakovic, B. (1999). Statistical Modeling by Wavelets. Wiley, New York.