The Annals of Statistics

Density estimation by wavelet thresholding

David L. Donoho,Iain M. Johnstone,Gérard Kerkyacharian, and Dominique Picard

Full-text: Open access


Density estimation is a commonly used test case for nonparametric estimation methods. We explore the asymptotic properties of estimators based on thresholding of empirical wavelet coefficients. Minimax rates of convergence are studied over a large range of Besov function classes $B_{\sigma pq}$ and for a range of global $L'_p$ error measures, $1 \leq p' < \infty$. A single wavelet threshold estimator is asymptotically minimax within logarithmic terms simultaneously over a range of spaces and error measures. In particular, when $p' > p$, some form of nonlinearity is essential, since the minimax linear estimators are suboptimal by polynomial powers of n. A second approach, using an approximation of a Gaussian white-noise model in a Mallows metric, is used to attain exactly optimal rates of convergence for quadratic error $(p' = 2)$.

Article information

Ann. Statist. Volume 24, Number 2 (1996), 508-539.

First available: 24 September 2002

Permanent link to this document

Mathematical Reviews number (MathSciNet)

Digital Object Identifier

Zentralblatt MATH identifier

Primary: 62G07: Density estimation 62G20: Asymptotic properties

Minimax estimation adaptive estimation density estimation spatial adaptation wavelet orthonormal bases Besov spaces


Donoho, David L.; Johnstone, Iain M.; Kerkyacharian, Gérard; Picard, Dominique. Density estimation by wavelet thresholding. The Annals of Statistics 24 (1996), no. 2, 508--539. doi:10.1214/aos/1032894451.

Export citation


  • Bergh, J. and L ¨ofstr ¨om, J. (1976). Interpolation Spaces-An Introduction. Springer, New York.
  • Birg´e, L. and Massart, P. (1996). From model selection to adaptive estimation. In Festschrift for Lucien Le Cam (D. Pollard and G. Yang, eds.). Springer, New York.
  • Daubechies, I. (1992). Ten Lectures on Wavelets. SIAM, Philadelphia.
  • Dely on, B. and Juditsky, A. (1993). Wavelet estimators, global error measures: revisited. Technical Report 782, Institut de Recherche en Informatique et Sy st emes Al´eatoires, Campus de Beaulieu.
  • Devroy e, L. and Gy ¨orf, L. (1985). Nonparametric Density Estimation, The L1 View. Wiley, New York.
  • Donoho, D. L. and Johnstone, I. M. (1994). Minimax risk over p-balls for q-error. Probab. Theory Related Fields 99 277-303.
  • Donoho, D. L. and Johnstone, I. M. (1996). Minimax estimation via wavelet shrinkage. Unpublished manuscript.
  • Donoho, D. L., Johnstone, I. M., Kerky acharian, G. and Picard, D. (1996). Universal near minimaxity of wavelet shrinkage. In Festschrift for Lucien Le Cam (D. Pollard and G. Yang, eds.). Springer, New York.
  • Donoho, D. L., Johnstone, I. M., Kerky acharian, G. and Picard, D. (1995). Wavelet shrinkage: asy mptopia? (with discussion). J. Roy. Statist. Soc. Ser. B 57 301-369.
  • Donoho, D. L., Liu, R. C. and MacGibbon, K. B. (1990). Minimax risk over hy perrectangles, and implications. Ann. Statist. 18 1416-1437.
  • Doukhan, P. and Leon, J. (1990). D´eviation quadratique d'estimateurs d'une densit´e par projection orthogonale. C. R. Acad. Sci. Paris S´er. I Math. 310 425-430.
  • Feller, W. (1971). An Introduction to Probability Theory and Its Applications 2. Wiley, New York.
  • Fix, G. and Strang, G. (1969). A Fourier analysis of the finite element method. Stud. Appl. Math. 48 265-273.
  • Frazier, M., Jawerth, B. and Weiss, G. (1991). Littlewood-Paley Theory and the Study of Function Spaces. Amer. Math. Soc., Providence, RI.
  • H¨ardle, W., Kerky acharian, G., Picard, D. and Tsy bakov, A. (1996). Wavelets and econometric applications. Technical report, Humboldt Univ., Berlin.
  • Johnstone, I., Kerky acharian, G. and Picard, D. (1992). Estimation d'une densit´e de probabilit´e par m´ethode d'ondelettes. C. R. Acad. Sci. Paris S´er. I Math. 315 211-216.
  • Kerky acharian, G. and Picard, D. (1992). Density estimation in Besov spaces. Statist. Probab. Lett. 13 15-24.
  • Kerky acharian, G. and Picard, D. (1993). Density estimation by kernel and wavelet methods: optimality of Besov spaces. Statist. Probab. Lett. 18 327-336.
  • Meyer, Y. (1990). Ondelettes et Op´erateurs, I: Ondelettes, II: Op´erateurs de Calder´on-Zy gmund, III: (with R. Coifman), Op´erateurs Multilin´eaires. Hermann, Paris. (English translation of first volume published by Cambridge Univ. Press.)
  • Nemirovskii, A. (1985). Nonparametric estimation of smooth regression function. Izv. Akad. Nauk. SSSR Tekhn. Kibernet. 3 50-60 (in Russian); Soviet J. Comput. Sy stems Sci. 23 1-11 (1986) (in English).
  • Nemirovskii, A., Poly ak, B. and Tsy bakov, A. (1985). Rate of convergence of nonparametric estimates of maximum-likelihood ty pe. Problems Inform. Transmission 21 258-272.
  • Nussbaum, M. (1995). Personal communication.
  • Peetre, J. (1975). New Thoughts on Besov Spaces. Dept. Mathematics, Duke Univ.
  • Petrov, V. V. (1975). Sums of Independent Random Variables. Springer, New York.
  • Pollard, D. (1984). Convergence of Stochastic Processes. Springer, New York.
  • Rosenthal, H. P. (1972). On the span in lp of sequences of independent random variables. Israel J. Math. 8 273-303.
  • Sakhanenko, A. I. (1991). Berry-Esseen ty pe estimates for large deviation probabilities. Siberian Math. J. 32 647-656.
  • Scott, D. (1992). Multivariate Density Estimation. Wiley, New York.
  • Silverman, B. W. (1986). Density Estimation for Statistics and Data Analy sis. Chapman and Hall, London.
  • Triebel, H. (1992). Theory of Function Spaces 2. Birkh¨auser, Basel.
  • Walter, G. (1992). Approximation of the delta function by wavelets. J. Approx. Theory 71 329-343.