The Annals of Statistics

Density estimation by wavelet thresholding

David L. Donoho, Iain M. Johnstone, Gérard Kerkyacharian, and Dominique Picard
Source: Ann. Statist. Volume 24, Number 2 (1996), 508-539.

Abstract

Density estimation is a commonly used test case for nonparametric estimation methods. We explore the asymptotic properties of estimators based on thresholding of empirical wavelet coefficients. Minimax rates of convergence are studied over a large range of Besov function classes $B_{\sigma pq}$ and for a range of global $L'_p$ error measures, $1 \leq p' < \infty$. A single wavelet threshold estimator is asymptotically minimax within logarithmic terms simultaneously over a range of spaces and error measures. In particular, when $p' > p$, some form of nonlinearity is essential, since the minimax linear estimators are suboptimal by polynomial powers of n. A second approach, using an approximation of a Gaussian white-noise model in a Mallows metric, is used to attain exactly optimal rates of convergence for quadratic error $(p' = 2)$.

First Page: Show Hide
Primary Subjects: 62G07, 62G20
Full-text: Open access
Links and Identifiers

Permanent link to this document: http://projecteuclid.org/euclid.aos/1032894451
Mathematical Reviews number (MathSciNet): MR1394974
Digital Object Identifier: doi:10.1214/aos/1032894451
Zentralblatt MATH identifier: 0860.62032

References

Bergh, J. and L ¨ofstr ¨om, J. (1976). Interpolation Spaces-An Introduction. Springer, New York.
Mathematical Reviews (MathSciNet): MR58:2349
Zentralblatt MATH: 0344.46071
Birg´e, L. and Massart, P. (1996). From model selection to adaptive estimation. In Festschrift for Lucien Le Cam (D. Pollard and G. Yang, eds.). Springer, New York.
Daubechies, I. (1992). Ten Lectures on Wavelets. SIAM, Philadelphia.
Mathematical Reviews (MathSciNet): MR93e:42045
Dely on, B. and Juditsky, A. (1993). Wavelet estimators, global error measures: revisited. Technical Report 782, Institut de Recherche en Informatique et Sy st emes Al´eatoires, Campus de Beaulieu.
Devroy e, L. and Gy ¨orf, L. (1985). Nonparametric Density Estimation, The L1 View. Wiley, New York.
Mathematical Reviews (MathSciNet): MR86i:62065
Zentralblatt MATH: 0546.62015
Donoho, D. L. and Johnstone, I. M. (1994). Minimax risk over p-balls for q-error. Probab. Theory Related Fields 99 277-303.
Mathematical Reviews (MathSciNet): MR95g:62019
Zentralblatt MATH: 0802.62006
Digital Object Identifier: doi:10.1007/BF01199026
Donoho, D. L. and Johnstone, I. M. (1996). Minimax estimation via wavelet shrinkage. Unpublished manuscript.
Donoho, D. L., Johnstone, I. M., Kerky acharian, G. and Picard, D. (1996). Universal near minimaxity of wavelet shrinkage. In Festschrift for Lucien Le Cam (D. Pollard and G. Yang, eds.). Springer, New York.
Mathematical Reviews (MathSciNet): MR1462946
Zentralblatt MATH: 0891.62025
Donoho, D. L., Johnstone, I. M., Kerky acharian, G. and Picard, D. (1995). Wavelet shrinkage: asy mptopia? (with discussion). J. Roy. Statist. Soc. Ser. B 57 301-369.
Mathematical Reviews (MathSciNet): MR96g:62068
Donoho, D. L., Liu, R. C. and MacGibbon, K. B. (1990). Minimax risk over hy perrectangles, and implications. Ann. Statist. 18 1416-1437.
Mathematical Reviews (MathSciNet): MR91k:62012
Zentralblatt MATH: 0705.62018
Digital Object Identifier: doi:10.1214/aos/1176347758
Project Euclid: euclid.aos/1176347758
Doukhan, P. and Leon, J. (1990). D´eviation quadratique d'estimateurs d'une densit´e par projection orthogonale. C. R. Acad. Sci. Paris S´er. I Math. 310 425-430.
Feller, W. (1971). An Introduction to Probability Theory and Its Applications 2. Wiley, New York.
Mathematical Reviews (MathSciNet): MR42:5292
Fix, G. and Strang, G. (1969). A Fourier analysis of the finite element method. Stud. Appl. Math. 48 265-273.
Mathematical Reviews (MathSciNet): MR258297
Zentralblatt MATH: 0179.22501
Frazier, M., Jawerth, B. and Weiss, G. (1991). Littlewood-Paley Theory and the Study of Function Spaces. Amer. Math. Soc., Providence, RI.
Mathematical Reviews (MathSciNet): MR92m:42021
Zentralblatt MATH: 0757.42006
H¨ardle, W., Kerky acharian, G., Picard, D. and Tsy bakov, A. (1996). Wavelets and econometric applications. Technical report, Humboldt Univ., Berlin.
Johnstone, I., Kerky acharian, G. and Picard, D. (1992). Estimation d'une densit´e de probabilit´e par m´ethode d'ondelettes. C. R. Acad. Sci. Paris S´er. I Math. 315 211-216.
Kerky acharian, G. and Picard, D. (1992). Density estimation in Besov spaces. Statist. Probab. Lett. 13 15-24.
Mathematical Reviews (MathSciNet): MR93a:62060
Zentralblatt MATH: 0749.62026
Kerky acharian, G. and Picard, D. (1993). Density estimation by kernel and wavelet methods: optimality of Besov spaces. Statist. Probab. Lett. 18 327-336.
Mathematical Reviews (MathSciNet): MR94j:62096
Zentralblatt MATH: 0793.62019
Meyer, Y. (1990). Ondelettes et Op´erateurs, I: Ondelettes, II: Op´erateurs de Calder´on-Zy gmund, III: (with R. Coifman), Op´erateurs Multilin´eaires. Hermann, Paris. (English translation of first volume published by Cambridge Univ. Press.)
Nemirovskii, A. (1985). Nonparametric estimation of smooth regression function. Izv. Akad. Nauk. SSSR Tekhn. Kibernet. 3 50-60 (in Russian); Soviet J. Comput. Sy stems Sci. 23 1-11 (1986) (in English).
Mathematical Reviews (MathSciNet): MR87k:62070
Nemirovskii, A., Poly ak, B. and Tsy bakov, A. (1985). Rate of convergence of nonparametric estimates of maximum-likelihood ty pe. Problems Inform. Transmission 21 258-272.
Mathematical Reviews (MathSciNet): MR87j:62071
Nussbaum, M. (1995). Personal communication.
Peetre, J. (1975). New Thoughts on Besov Spaces. Dept. Mathematics, Duke Univ.
Mathematical Reviews (MathSciNet): MR461123
Zentralblatt MATH: 0356.46038
Petrov, V. V. (1975). Sums of Independent Random Variables. Springer, New York.
Mathematical Reviews (MathSciNet): MR388499
Pollard, D. (1984). Convergence of Stochastic Processes. Springer, New York.
Mathematical Reviews (MathSciNet): MR86i:60074
Zentralblatt MATH: 0544.60045
Rosenthal, H. P. (1972). On the span in lp of sequences of independent random variables. Israel J. Math. 8 273-303.
Mathematical Reviews (MathSciNet): MR271721
Zentralblatt MATH: 0213.19303
Digital Object Identifier: doi:10.1007/BF02771562
Sakhanenko, A. I. (1991). Berry-Esseen ty pe estimates for large deviation probabilities. Siberian Math. J. 32 647-656.
Mathematical Reviews (MathSciNet): MR1142075
Scott, D. (1992). Multivariate Density Estimation. Wiley, New York.
Mathematical Reviews (MathSciNet): MR1191168
Silverman, B. W. (1986). Density Estimation for Statistics and Data Analy sis. Chapman and Hall, London.
Mathematical Reviews (MathSciNet): MR87k:62074
Zentralblatt MATH: 0617.62042
Triebel, H. (1992). Theory of Function Spaces 2. Birkh¨auser, Basel.
Mathematical Reviews (MathSciNet): MR93f:46029
Zentralblatt MATH: 0763.46025
Walter, G. (1992). Approximation of the delta function by wavelets. J. Approx. Theory 71 329-343.
Zentralblatt MATH: 0766.41020
Mathematical Reviews (MathSciNet): MR1191579
Digital Object Identifier: doi:10.1016/0021-9045(92)90123-6

2012 © Institute of Mathematical Statistics

The Annals of Statistics

The Annals of Statistics