General empirical Bayes wavelet methods and exactly adaptive minimax estimation
Cun-Hui Zhang
Source: Ann. Statist. Volume 33, Number 1 (2005), 54-100.
Abstract
In many statistical problems, stochastic signals can be represented as a sequence of noisy wavelet coefficients. In this paper, we develop general empirical Bayes methods for the estimation of true signal. Our estimators approximate certain oracle separable rules and achieve adaptation to ideal risks and exact minimax risks in broad collections of classes of signals. In particular, our estimators are uniformly adaptive to the minimum risk of separable estimators and the exact minimax risks simultaneously in Besov balls of all smoothness and shape indices, and they are uniformly superefficient in convergence rates in all compact sets in Besov spaces with a finite secondary shape parameter. Furthermore, in classes nested between Besov balls of the same smoothness index, our estimators dominate threshold and James–Stein estimators within an infinitesimal fraction of the minimax risks. More general block empirical Bayes estimators are developed. Both white noise with drift and nonparametric regression are considered.
Primary Subjects: 62C12, 62G05, 62G08, 62G20, 62C25
Keywords: Empirical Bayes; wavelet; adaptation; minimax estimation; white noise; nonparametric regression; threshold estimate; Besov space
Full-text: Open access
Links and Identifiers
Permanent link to this document: http://projecteuclid.org/euclid.aos/1112967699
Digital Object Identifier: doi:10.1214/009053604000000995
Zentralblatt MATH identifier:
02182556
Mathematical Reviews number (MathSciNet):
MR2157796
References
Abramovich, F., Benjamini, Y., Donoho, D. L. and Johnstone, I. M. (2000). Adapting to unknown sparsity by controlling the false discovery rate. Technical Report 2000-19, Dept. Statistics, Stanford Univ.
Barron, A., Birgé, L. and Massart, P. (1999). Risk bounds for model selection via penalization. Probab. Theory Related Fields 113 301--413.
Mathematical Reviews (MathSciNet):
MR1679028
Digital Object Identifier: doi:10.1007/s004400050210
Berger, J. O. (1985). Statistical Decision Theory and Bayesian Analysis. Springer, New York.
Mathematical Reviews (MathSciNet):
MR804611
Zentralblatt MATH:
0572.62008
Breiman, L., Friedman, J., Olshen, R. and Stone, C. (1984). Classification and Regression Trees. Wadsworth, Belmont, CA.
Mathematical Reviews (MathSciNet):
MR726392
Zentralblatt MATH:
0541.62042
Brown, L. D., Cai, T. T., Low, M. and Zhang, C.-H. (2002). Asymptotic equivalence theory for nonparametric regression with random design. Ann. Statist. 30 688--707.
Mathematical Reviews (MathSciNet):
MR1922538
Digital Object Identifier: doi:10.1214/aos/1028674838
Project Euclid: euclid.aos/1028674838
Brown, L. D. and Low, M. G. (1996). Asymptotic equivalence of nonparametric regression and white noise. Ann. Statist. 24 2384--2398.
Mathematical Reviews (MathSciNet):
MR1425958
Digital Object Identifier: doi:10.1214/aos/1032181159
Project Euclid: euclid.aos/1032181159
Brown, L. D., Low, M. G. and Zhao, L. H. (1997). Superefficiency in nonparametric function estimation. Ann. Statist. 25 2607--2625.
Mathematical Reviews (MathSciNet):
MR1604424
Digital Object Identifier: doi:10.1214/aos/1030741087
Project Euclid: euclid.aos/1030741087
Cai, T. T. (1999). Adaptive wavelet estimation: A block thresholding and oracle inequality approach. Ann. Statist. 27 898--924.
Mathematical Reviews (MathSciNet):
MR1724035
Digital Object Identifier: doi:10.1214/aos/1018031262
Project Euclid: euclid.aos/1018031262
Cai, T. T. (2000). On adaptability and information-pooling in nonparametric function estimation. Technical report, Dept. Statistics, Univ. Pennsylvania.
Cavalier, L. and Tsybakov, A. B. (2001). Penalized blockwise Stein's method, monotone oracles and sharp adaptive estimation. Math. Methods Statist. 10 247--282.
Mathematical Reviews (MathSciNet):
MR1867161
Cavalier, L. and Tsybakov, A. B. (2002). Sharp adaptation for inverse problems with random noise. Probab. Theory Related Fields 123 323--354.
Mathematical Reviews (MathSciNet):
MR1918537
Digital Object Identifier: doi:10.1007/s004400100169
Chui, C. K. (1992). An Introduction to Wavelets. Academic Press, San Diego, CA.
Mathematical Reviews (MathSciNet):
MR1150048
Zentralblatt MATH:
0925.42016
Daubechies, I. (1992). Ten Lectures on Wavelets. SIAM, Philadelphia.
Mathematical Reviews (MathSciNet):
MR1162107
Zentralblatt MATH:
0776.42018
Donoho, D. L. and Johnstone, I. M. (1994a). Ideal spatial adaptation via wavelet shrinkage. Biometrika 81 425--455.
Mathematical Reviews (MathSciNet):
MR1311089
Zentralblatt MATH:
0815.62019
Donoho, D. L. and Johnstone, I. M. (1994b). Minimax risk over $\ell_p$-balls for $\ell_q$-error. Probab. Theory Related Fields 99 277--303.
Mathematical Reviews (MathSciNet):
MR1278886
Digital Object Identifier: doi:10.1007/BF01199026
Donoho, D. L. and Johnstone, I. M. (1995). Adapting to unknown smoothness via wavelet shrinkage. J. Amer. Statist. Assoc. 90 1200--1224.
Mathematical Reviews (MathSciNet):
MR1379464
Donoho, D. L. and Johnstone, I. M. (1998). Minimax estimation via wavelet shrinkage. Ann. Statist. 26 879--921.
Mathematical Reviews (MathSciNet):
MR1635414
Digital Object Identifier: doi:10.1214/aos/1024691081
Project Euclid: euclid.aos/1024691081
Donoho, D. L., Johnstone, I. M., Kerkyacharian, G. and Picard, D. (1995). Wavelet shrinkage: Asymptopia? (with discussion). J. Roy. Statist. Soc. Ser. B 57 301--369.
Mathematical Reviews (MathSciNet):
MR1323344
Donoho, D. L., Johnstone, I. M., Kerkyacharian, G. and Picard, D. (1996). Density estimation by wavelet thresholding. Ann. Statist. 24 508--539.
Mathematical Reviews (MathSciNet):
MR1394974
Digital Object Identifier: doi:10.1214/aos/1032894451
Project Euclid: euclid.aos/1032894451
Dubuc, S. (1986). Interpolation through an iterative scheme. J. Math. Anal. Appl. 114 185--204.
Mathematical Reviews (MathSciNet):
MR829123
Digital Object Identifier: doi:10.1016/0022-247X(86)90077-6
Efromovich, S. (1985). Nonparametric estimation of a density of unknown smoothness. Theory Probab. Appl. 30 557--568.
Mathematical Reviews (MathSciNet):
MR805304
Efromovich, S. (1999). Nonparametric Curve Estimation. Springer, New York.
Mathematical Reviews (MathSciNet):
MR1705298
Zentralblatt MATH:
0935.62039
Efromovich, S. and Pinsker, M. S. (1984). An adaptive algorithm for nonparametric filtering. Automat. Remote Control 11 58--65.
Efromovich, S. and Pinsker, M. S. (1986). An adaptive algorithm for minimax nonparametric estimation of a spectral density. Problems Inform. Transmission 22 62--76.
Mathematical Reviews (MathSciNet):
MR865599
Efron, B. and Morris, C. (1973). Stein's estimation rule and its competitors---an empirical Bayes approach. J. Amer. Statist. Assoc. 68 117--130.
Mathematical Reviews (MathSciNet):
MR388597
Foster, D. P. and George, E. I. (1994). The risk inflation criterion for multiple regression. Ann. Statist. 22 1947--1975.
Mathematical Reviews (MathSciNet):
MR1329177
JSTOR: links.jstor.org
Foster, D. P., Stine, R. A. and Wyner, A. J. (2002). Universal codes for finite sequences of integers drawn from a monotone distribution. IEEE Trans. Inform. Theory 48 1713--1720.
Mathematical Reviews (MathSciNet):
MR1909485
Digital Object Identifier: doi:10.1109/TIT.2002.1003851
Friedman, J. (1991). Multivariate adaptive regression splines (with discussion). Ann. Statist. 19 1--141.
Mathematical Reviews (MathSciNet):
MR1091842
JSTOR: links.jstor.org
Friedman, J. and Silverman, B. W. (1989). Flexible parsimonious smoothing and additive modeling. Technometrics 31 3--21.
Mathematical Reviews (MathSciNet):
MR997668
Golubev, G. K. (1992). Nonparametric estimation of smooth probability densities in $L_2$. Problems Inform. Transmission 28 44--54.
Mathematical Reviews (MathSciNet):
MR1163140
Grama, I. and Nussbaum, M. (1998). Asymptotic equivalence for nonparametric generalized linear models. Probab. Theory Related Fields 111 167--214.
Mathematical Reviews (MathSciNet):
MR1633574
Digital Object Identifier: doi:10.1007/s004400050166
Hall, P., Kerkyacharian, G. and Picard, D. (1998). Block threshold rules for curve estimation using kernel and wavelet methods. Ann. Statist. 26 922--942.
Mathematical Reviews (MathSciNet):
MR1635418
Digital Object Identifier: doi:10.1214/aos/1024691082
Project Euclid: euclid.aos/1024691082
Hall, P., Kerkyacharian, G. and Picard, D. (1999). On the minimax optimality of block thresholding wavelet estimators. Statist. Sinica 9 33--49.
Mathematical Reviews (MathSciNet):
MR1678880
Hall, P. and Patil, P. (1995). Formulae for mean integrated squared error of nonlinear wavelet-based density estimators. Ann. Statist. 23 905--928.
Mathematical Reviews (MathSciNet):
MR1345206
JSTOR: links.jstor.org
Hall, P. and Patil, P. (1996). Effect of threshold rules on performance of wavelet-based curve estimators. Statist. Sinica 6 331--345.
Mathematical Reviews (MathSciNet):
MR1399306
Härdle, W., Kerkyacharian, G., Picard, D. and Tsybakov, A. (1998). Wavelets, Approximation, and Statistical Applications. Lecture Notes in Statist. 129. Springer, New York.
Mathematical Reviews (MathSciNet):
MR1618204
Hastie, T., Tibshirani, R. and Friedman, J. H. (2001). The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer, New York.
Mathematical Reviews (MathSciNet):
MR1851606
Zentralblatt MATH:
0973.62007
Ibragimov, I. A. and Khas'minskii, R. Z. (1981). Statistical Estimation: Asymptotic Theory. Springer, New York.
Mathematical Reviews (MathSciNet):
MR620321
Zentralblatt MATH:
0467.62026
James, W. and Stein, C. (1961). Estimation with quadratic loss. Proc. Fourth Berkeley Symp. Math. Statist. Probab. 1 361--379. Univ. California Press, Berkeley.
Mathematical Reviews (MathSciNet):
MR133191
Johnstone, I. M., Kerkyacharian, G. and Picard, D. (1992). Estimation d'une densité de probabilité par méthode d'ondelettes. C. R. Acad. Sci. Paris Sér. I Math. 315 211--216.
Mathematical Reviews (MathSciNet):
MR1197240
Johnstone, I. M. and Silverman, B. W. (1998). Empirical Bayes approaches to mixture problems and wavelet regression. Technical report, Dept. Statistics, Stanford Univ. Available at http://www-stat.stanford.edu/people/faculty/johnstone/techreports.html.
Johnstone, I. M. and Silverman, B. W. (2004). Needles and hay in haystacks: Empirical Bayes estimates of possibly sparse sequences. Ann. Statist. 32 1594--1649.
Mathematical Reviews (MathSciNet):
MR2089135
Digital Object Identifier: doi:10.1214/009053604000000030
Project Euclid: euclid.aos/1091626180
Johnstone, I. M. and Silverman, B. W. (2005). Empirical Bayes selection of wavelet thresholds. Ann. Statist. 33. To appear.
Mathematical Reviews (MathSciNet):
MR2166560
Digital Object Identifier: doi:10.1214/009053605000000345
Project Euclid: euclid.aos/1123250227
Juditsky, A. (1997). Wavelet estimators: Adapting to unknown smoothness. Math. Methods Statist. 6 1--25.
Mathematical Reviews (MathSciNet):
MR1456644
Lepski, O. V., Mammen, E. and Spokoiny, V. G. (1997). Optimal spatial adaptation to inhomogeneous smoothness: An approach based on kernel estimates with variable bandwidth selectors. Ann. Statist. 25 929--947.
Mathematical Reviews (MathSciNet):
MR1447734
Digital Object Identifier: doi:10.1214/aos/1069362731
Project Euclid: euclid.aos/1069362731
Morris, C. N. (1983). Parametric empirical Bayes inference: Theory and applications (with discussion). J. Amer. Statist. Assoc. 78 47--65.
Mathematical Reviews (MathSciNet):
MR696849
Nussbaum, M. (1996). Asymptotic equivalence of density estimation and Gaussian white noise. Ann. Statist. 24 2399--2430.
Mathematical Reviews (MathSciNet):
MR1425959
Digital Object Identifier: doi:10.1214/aos/1032181160
Project Euclid: euclid.aos/1032181160
Robbins, H. (1951). Asymptotically subminimax solutions of compound statistical decision problems. Proc. Second Berkeley Symp. Math. Statist. Probab. 131--148. Univ. California Press, Berkeley.
Mathematical Reviews (MathSciNet):
MR44803
Zentralblatt MATH:
0044.14803
Robbins, H. (1956). An empirical Bayes approach to statistics. Proc. Third Berkeley Symp. Math. Statist. Probab. 1 157--163. Univ. California Press, Berkeley.
Mathematical Reviews (MathSciNet):
MR84919
Zentralblatt MATH:
0074.35302
Robbins, H. (1983). Some thoughts on empirical Bayes estimation. Ann. Statist. 11 713--723.
Mathematical Reviews (MathSciNet):
MR707923
JSTOR: links.jstor.org
Stein, C. (1956). Inadmissibility of the usual estimator for the mean of a multivariate normal distribution. Proc. Third Berkeley Symp. Math. Statist. Probab. 1 197--206. Univ. California Press, Berkeley.
Mathematical Reviews (MathSciNet):
MR84922
Zentralblatt MATH:
0073.35602
Stein, C. (1981). Estimation of the mean of a multivariate normal distribution. Ann. Statist. 9 1135--1151.
Mathematical Reviews (MathSciNet):
MR630098
JSTOR: links.jstor.org
Stone, C. J. (1994). The use of polynomial splines and their tensor products in multivariate function estimation (with discussion). Ann. Statist. 22 118--184.
Mathematical Reviews (MathSciNet):
MR1272079
JSTOR: links.jstor.org
Zhang, C.-H. (1990). Fourier methods for estimating mixing densities and distributions. Ann. Statist. 18 806--831.
Mathematical Reviews (MathSciNet):
MR1056338
JSTOR: links.jstor.org
Zhang, C.-H. (1997). Empirical Bayes and compound estimation of normal means. Statist. Sinica 7 181--193.
Mathematical Reviews (MathSciNet):
MR1441153
Zhang, C.-H. (2000). General empirical Bayes wavelet methods. Technical Report 2000-007, Dept. Statistics, Rutgers Univ. Available at http://stat.rutgers.edu/~cunhui/papers.
The Annals of Statistics
