The Annals of Statistics

Minimax estimation with thresholding and its application to wavelet analysis

Harrison H. Zhou and J. T. Gene Hwang

Full-text: Open access

Abstract

Many statistical practices involve choosing between a full model and reduced models where some coefficients are reduced to zero. Data were used to select a model with estimated coefficients. Is it possible to do so and still come up with an estimator always better than the traditional estimator based on the full model? The James–Stein estimator is such an estimator, having a property called minimaxity. However, the estimator considers only one reduced model, namely the origin. Hence it reduces no coefficient estimator to zero or every coefficient estimator to zero. In many applications including wavelet analysis, what should be more desirable is to reduce to zero only the estimators smaller than a threshold, called thresholding in this paper. Is it possible to construct this kind of estimators which are minimax?

In this paper, we construct such minimax estimators which perform thresholding. We apply our recommended estimator to the wavelet analysis and show that it performs the best among the well-known estimators aiming simultaneously at estimation and model selection. Some of our estimators are also shown to be asymptotically optimal.

Article information

Source
Ann. Statist. Volume 33, Number 1 (2005), 101-125.

Dates
First available in Project Euclid: 8 April 2005

Permanent link to this document
https://projecteuclid.org/euclid.aos/1112967700

Digital Object Identifier
doi:10.1214/009053604000000977

Mathematical Reviews number (MathSciNet)
MR2157797

Zentralblatt MATH identifier
1064.62013

Subjects
Primary: 62G05: Estimation 62J07: Ridge regression; shrinkage estimators
Secondary: 62C10: Bayesian problems; characterization of Bayes procedures 62H25: Factor analysis and principal components; correspondence analysis

Keywords
James–Stein estimator model selection VisuShrink SureShrink BlockJS

Citation

Zhou, Harrison H.; Hwang, J. T. Gene. Minimax estimation with thresholding and its application to wavelet analysis. Ann. Statist. 33 (2005), no. 1, 101--125. doi:10.1214/009053604000000977. https://projecteuclid.org/euclid.aos/1112967700.


Export citation

References

  • Anderson, T. W. (1955). The integral of a symmetric unimodal function over a symmetric convex set and some probability inequalities. Proc. Amer. Math. Soc. 6 170--176.
  • Antoniadis, A. and Fan, J. (2001). Regularization of wavelet approximations (with discussion). J. Amer. Statist. Assoc. 96 939--967.
  • Antoniadis, A., Leporini, D. and Pesquet, J.-C. (2002). Wavelet thresholding for some classes of non-Gaussian noise. Statist. Neerlandica 56 434--453.
  • Beran, R. and Dümbgen, L. (1998). Modulation of estimators and confidence sets. Ann. Statist. 26 1826--1856.
  • Berger, J. (1976). Tail minimaxity in location vector problems and its applications. Ann. Statist. 4 33--50.
  • Berger, J. (1980). Improving on inadmissible estimators in continuous exponential families with applications to simultaneous estimation of gamma scale parameters. Ann. Statist. 8 545--571.
  • Brown, L. D. (1971). Admissible estimators, recurrent diffusions, and insoluble boundary value problems. Ann. Math. Statist. 42 855--903.
  • Cai, T. (1999). Adaptive wavelet estimation: A block thresholding and oracle inequality approach. Ann. Statist. 27 898--924.
  • Donoho, D. L. and Johnstone, I. (1994). Ideal spatial adaptation via wavelet shrinkage. Biometrika 81 425--455.
  • Donoho, D. L. and Johnstone, I. (1995). Adapting to unknown smoothness via wavelet shrinkage. J. Amer. Statist. Assoc. 90 1200--1224.
  • Gao, H.-Y. (1998). Wavelet shrinkage denoising using the non-negative garrote. J. Comput. Graph. Statist. 7 469--488.
  • Gauch, H. (1993). Prediction, parsimony and noise. American Scientist 81 468--478.
  • George, E. I. (1986a). Minimax multiple shrinkage estimation. Ann. Statist. 14 188--205.
  • George, E. I. (1986b). Combining minimax shrinkage estimators. J. Amer. Statist. Assoc. 81 437--445.
  • James, W. and Stein, C. (1961). Estimation with quadratic loss. Proc. Fourth Berkeley Symp. Math. Statist. Probab. 1 361--379. Univ. California Press, Berkeley.
  • Lehmann, E. L. (1983). Theory of Point Estimation. Wiley, New York
  • Lehmann, E. L. and Casella, G. C. (1998). Theory of Point Estimation, 2nd ed. Springer, New York.
  • Mallat, S. G. (1989). A theory for multiresolution signal decomposition: The wavelet representation. IEEE Trans. Pattern Analysis Machine Intelligence 11 674--693.
  • Stein, C. (1981). Estimation of the mean of a multivariate normal distribution. Ann. Statist. 9 1135--1151.
  • Vidakovic, B. (1999). Statistical Modeling by Wavelets. Wiley, New York.
  • Zhou, H. H. and Hwang, J. T. G. (2003). Minimax estimation with thresholding. Technical report, Cornell Statistical Center.