The Annals of Statistics

Adaptive wavelet estimation: a block thresholding and oracle inequality approach

T. Tony Cai

Full-text: Open access


We study wavelet function estimation via the approach of block thresholding and ideal adaptation with oracle. Oracle inequalities are derived and serve as guides for the selection of smoothing parameters. Based on an oracle inequality and motivated by the data compression and localization properties of wavelets, an adaptive wavelet estimator for nonparametric regression is proposed and the optimality of the procedure is investigated. We show that the estimator achieves simultaneously three objectives: adaptivity, spatial adaptivity and computational efficiency. Specifically, it is proved that the estimator attains the exact optimal rates of convergence over a range of Besov classes and the estimator achieves adaptive local minimax rate for estimating functions at a point. The estimator is easy to implement, at the computational cost of $O(n)$. Simulation shows that the estimator has excellent numerical performance relative to more traditional wavelet estimators.

Article information

Ann. Statist., Volume 27, Number 3 (1999), 898-924.

First available in Project Euclid: 5 April 2002

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G07: Density estimation
Secondary: 62G20: Asymptotic properties

Adaptivity Besov space block thresholding James-Stein estimator nonparametric regression oracle inequality spatial adaptivity wavelets


Cai, T. Tony. Adaptive wavelet estimation: a block thresholding and oracle inequality approach. Ann. Statist. 27 (1999), no. 3, 898--924. doi:10.1214/aos/1018031262.

Export citation


  • Berger, J. O. (1985). Statistical Decision Theory and Bayesian Analysis, 2nd ed. Springer, New York. Brown, L. D. and Low, M. G. (1996a). Asymptotic equivalence of nonparametric regression and white noise. Ann. Statist. 24 2384-2398. Brown, L. D. and Low, M. G. (1996b). A constrained risk inequality with applications to nonparametric functional estimations. Ann. Statist. 24 2524-2535.
  • Cai, T. (1998). Numerical comparisons of BlockJS estimator with conventional wavelet methods. Unpublished manuscript.
  • Cai, T. and Brown, L. D. (1998). Wavelet shrinkage for nonequispaced samples. Ann. Statist. 26 1783-1799.
  • Chambolle, A., DeVore, R., Lee, N. and Lucier, B. (1998). Nonlinear wavelet image processing: variational problems, compression, and noise removal through wavelet shrinkage. IEEE Trans. Image Processing 70 319-335.
  • Coifman, R. R. and Donoho, D. L. (1995). Translation invariant denoising. Wavelets and Statistics. Lecture Notes in Statist. 103 125-150. Springer, New York.
  • Daubechies, I. (1992). Ten Lectures on Wavelets. SIAM, Philadelphia.
  • DeVore, R., Jawerth, B. and Popov, V. (1992). Compression of wavelet decompositions. Amer. J. Math. 114 737-785.
  • DeVore, R. and Popov, V. (1988). Interpolation of Besov spaces. Trans. Amer. Math. Soc. 305 397-414.
  • Donoho, D. L. and Johnstone, I. M. (1994). Ideal spatial adaptation via wavelet shrinkage. Biometrika 81 425-455.
  • Donoho, D. L. and Johnstone, I. M. (1995). Adapt to unknown smoothness via wavelet shrinkage. J. Amer. Statist. Assoc. 90 1200-1224.
  • Donoho, D. L. and Johnstone, I. M. (1998). Minimax estimation via wavelet shrinkage. Ann. Statist. 26 879-921.
  • Donoho, D. L., Johnstone, I. M., Kerkyacharian, G. and Picard, D. (1995). Wavelet shrinkage: asymptopia? J. Roy. Statist. Soc. Ser. B 57 301-369.
  • Efromovich, S. Y. and Pinsker, M. S. (1984). Self learning algorithm of nonparametric filtration. Automat. i Telemeh. 11 58-65. (In Russian.)
  • Efromovich, S. Y. (1998). Simultaneous sharp estimation of functions and their derivatives. Ann. Statist. 26 273-278.
  • Efron, B. and Morris, C. (1973). Stein's estimation rule and its competitors-an empirical Bayes approach. J. Amer. Statist. Assoc. 68 117-130.
  • Gao, H.-Y. (1998). Wavelet shrinkage denoising using the non-negative garrote. J. Comput. Graph. Statist. 7 469-488.
  • Hall, P., Kerkyacharian, G. and Picard, D. (1999). On the minimax optimality of block thresholded wavelet estimators. Statist. Sinica 9 33-50.
  • Hall, P., Penev, S., Kerkyacharian, G. and Picard, D. (1997). Numerical performance of block thresholded wavelet estimators. Statist. Comput. 7 115-124.
  • James, W. and Stein, C. (1961). Estimation with quadratic loss. Proc. Fourth Berkeley Symp. Math. Statist. Probab. 1 361-380. Univ. California Press, Berkeley.
  • Lehmann, E. L. (1983). Theory of Point Estimation. Wiley, New York.
  • Lepski, O. V. (1990). On a problem of adaptive estimation on white Gaussian noise. Theory Probab. Appl. 35 454-466.
  • Marron, J. S., Adak, S., Johnstone, I. M., Neumann, M. H. and Patil, P. (1998). Exact risk analysis of wavelet regression. J. Comput. Graph. Statist. 7 278-309.
  • Meyer, Y. (1992). Wavelets and Operators. Cambridge Univ. Press.
  • Stein, C. (1955). Inadmissibility of the usual estimator for the mean of a multivariate normal distribution. Proc. Third Berkeley Symp. Math. Stat. Probab. 1 197-206. Univ. California Press, Berkeley.
  • Stein, C. (1981). Estimation of the mean of a multivariate normal distribution. Ann. Statist. 9 1135-1151.
  • Strang, G. (1992). Wavelet and dilation equations: a brief introduction. SIAM Rev. 31 614-627.
  • Wahba, G. (1990). Spline Models for Observational Data. SIAM, Philadelphia.