Statistics Surveys

Wavelet methods in statistics: some recent developments and their applications

Anestis Antoniadis

Full-text: Open access

Abstract

The development of wavelet theory has in recent years spawned applications in signal processing, in fast algorithms for integral transforms, and in image and function representation methods. This last application has stimulated interest in wavelet applications to statistics and to the analysis of experimental data, with many successes in the efficient analysis, processing, and compression of noisy signals and images.

This is a selective review article that attempts to synthesize some recent work on “nonlinear” wavelet methods in nonparametric curve estimation and their role on a variety of applications. After a short introduction to wavelet theory, we discuss in detail several wavelet shrinkage and wavelet thresholding estimators, scattered in the literature and developed, under more or less standard settings, for density estimation from i.i.d. observations or to denoise data modeled as observations of a signal with additive noise. Most of these methods are fitted into the general concept of regularization with appropriately chosen penalty functions. A narrow range of applications in major areas of statistics is also discussed such as partial linear regression models and functional index models. The usefulness of all these methods are illustrated by means of simulations and practical examples.

Article information

Source
Statist. Surv., Volume 1 (2007), 16-55.

Dates
First available in Project Euclid: 3 December 2007

Permanent link to this document
https://projecteuclid.org/euclid.ssu/1196693422

Digital Object Identifier
doi:10.1214/07-SS014

Mathematical Reviews number (MathSciNet)
MR2520413

Zentralblatt MATH identifier
1300.62028

Subjects
Primary: 60K35: Interacting random processes; statistical mechanics type models; percolation theory [See also 82B43, 82C43] 60K35: Interacting random processes; statistical mechanics type models; percolation theory [See also 82B43, 82C43]
Secondary: 60K35: Interacting random processes; statistical mechanics type models; percolation theory [See also 82B43, 82C43]

Keywords
curve smoothing density estimation wavelet thresholding penalized least-squares robust regression partial linear models mixed effects models inverse regression time series prediction

Citation

Antoniadis, Anestis. Wavelet methods in statistics: some recent developments and their applications. Statist. Surv. 1 (2007), 16--55. doi:10.1214/07-SS014. https://projecteuclid.org/euclid.ssu/1196693422


Export citation

References

  • ABRAMOVICH, F., BAILEY, T. C. and SAPATINAS, T. (2000). Wavelet analysis and its statistical applications. The Statistician, 49, 1–29.
  • ALSBERG, B. K. (1993). Representation of spectra by continuous functions. Journal of Chemometrics, 7, 177–193.
  • AMATO, U., ANTONIADIS, A. and DE FEISS, I. (2006a). Dimension reduction in functional regression with applications. Comput. Statist. Data Anal., 50, 2422–2446.
  • AMATO, U., ANTONIADIS, A. and PENSKY, M. (2006b). Wavelet kernel penalized estimation for non-equispaced design regression. Statistics and Computing, 16, 37–56.
  • AMATO, U. and VUZA, D. (1997). Wavelet approximation of a function from samples affected by noise. Rev. Roumaine Math. Pure Appl., 42, 81–493.
  • ANTONIADIS, A. (1996). Smoothing noisy data with tapered coiflets series. Scand. J. Statist., 23, 313–330.
  • ANTONIADIS, A. (1997). Wavelets in statistics: a review (with discussion). J. Ital. Statist. Soc., 6.
  • ANTONIADIS, A. and FAN, J. (2001). Regularization of wavelets approximations. J. Amer. Statist. Assoc., 96, 939–967.
  • ANTONIADIS, A., GRÉGOIRE, G. and VIAL, P. (1997). Random design wavelet curve smoothing. Statist. Probab. Lett., 35, 225–232.
  • ANTONIADIS, A. and SAPATINAS, T. (2003). Wavelet methods for continuous-time prediction using representations of autoregressive processes in Hilbert spaces. J. of Multivariate Anal., 87, 133–158.
  • ANTONIADIS, A. and SAPATINAS, T. (2007). Estimation and inference in functional mixed-effects models. Comput. Statist. Data Anal., 51, 4793–4813.
  • BLACK, M., SAPIRO, G., MARIMONT, D. and HEEGER, D. (1998). Robust anisotropic diffusion. IEEE Transactions on Image Processing, 7, 421–432.
  • BROWN, L. D., CAI, T. and ZHOU, H. (2006). Robust nonparametric estimation via wavelet median regression. Technical report, University of, Pennsylvania.
  • BURRUS, C., GONIPATH, R. and GUO, H. (1998). Introduction to Wavelets and Wavelet Transforms: A Primer. Englewood Cliffs, Prentice, Hall.
  • CAI, T. (1999). Adaptive wavelet estimation: a block thresholding and oracle inequality approach. Ann. Statist., 27, 898–924.
  • CAI, T. (2001). Invited discussion on “Regularization of wavelets approximations” by A. Antoniadis and J. Fan. J. Amer. Statist. Assoc., 96, 960–962.
  • CAI, T. (2002). On block thresholding in wavelet regression: Adaptivity, block size, and threshold level. Statist. Sinica, 12, 1241–1273.
  • CAI, T. and BROWN, L. (1998). Wavelet shrinkage for nonequispaced samples. Ann. Statist., 26, 1783–1799.
  • CAI, T. and BROWN, L. (1999). Wavelet estimation for samples with random uniform design. Statist. Probab. Lett., 42, 313–321.
  • CAI, T. and SILVERMAN, B. W. (2001). Incorporating information on neighboring coefficients into wavelet estimation. Sankhya, Series B, 63, 127–148.
  • CAI, T. and ZHOU, H. (2005). A data-driven block thresholding approach to wavelet estimation. Technical report, Department of Statistics, University of Pennsylvania, http://stat.wharton.upenn.edu/, tcai/paper/.
  • CHANG, X. and QU, L. (2004). Wavelet estimation of partially linear models. Comput. Statist. Data Anal., 47, 31–48.
  • CHARBONNIER, P., BLANC-FÉRAUD, L., AUBERT, G. and BARLAUD, M. (1994). Two deterministic half-quadratic regularization algorithms for computed imaging. In Proc. 1994 IEEE International Conference on Image Processing, vol. 2. IEEE Computer Society Press, Austin, TX, 168–172.
  • CHICKEN, E. (2003). Block thresholding and wavelet estimation for nonequispaced samples. J. Statist. Plann. Inference, 116, 113–129.
  • CHICKEN, E. and CAI, T. (2005). Block thresholding for density estimation: Local and global adaptivity. J. Multivariate Analysis, 95, 76–106.
  • COIFMAN, R. and DONOHO, D. (1995). Translation-invariant de-noising. In Wavelets and Statistics (A. Antoniadis and G. Oppenheim, eds.), vol. 103. Springer-Verlag, New York, 125–150.
  • COOK, D. (2000). Save: A method for dimension reduction and graphics in regression. Communications in Statistics: Theory and Methods, 29, 161–175.
  • DAUBECHIES, I. (1992). Ten Lectures on Wavelets. SIAM, Philadelphia.
  • DENHAM, M. C. and BROWN, P. J. (1993). Calibration with many variables. Applied Statistics, 42, 515–528.
  • DONOHO, D., JOHNSTONE, I., KERKYACHARIAN, G. and PICARD, D. (1995). Wavelet shrinkage: asymptopia? (with discussion). J. Roy. Statist. Soc. Ser. B, 57, 301–337.
  • DONOHO, D. L. (1995). De-noising by soft thresholding. IEEE Transactions on Information Theory, 41, 613–627.
  • DONOHO, D. L. and JOHNSTONE, I. M. (1994). Ideal spatial adaptation by wavelet shrinkage. Biometrika, 81, 425–455.
  • DONOHO, D. L. and JOHNSTONE, I. M. (1998). Minimax estimation via wavelet shrinkage. Ann. Statist., 26, 879–921.
  • DROSKE, M. and RUMPF, M. (2004). A variational approach to non-rigid morphological registration. SIAM Appl. Math., 64, 668–687.
  • ENGLE, R., GRANGER, C., RICE, J. and WEISS, A. (1986). Semiparametric estimates of the relation between weather and electricity sales. J. Amer. Statist. Assoc., 81, 310–320.
  • EUBANK, R. (1999). Nonparametric Regression and Spline Smoothing. 2nd ed. Marcel Dekker, New, York.
  • FADILI, J. and BULLMORE, E. (2005). Penalized partially linear models using sparse representation with an application to fmri time series. IEEE Transactions on signal processing, 53, 3436–3448.
  • FAN, J. and GIJBELS, I. (1996). Local Polynomial Modelling and its Applications. Chapman & Hall, London.
  • FAN, J. and LI, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. J. Amer. Statist. Assoc., 96, 1348–1360.
  • FOSTER, G. (1996). Wavelets for period analysis of unevenly sampled time series. The Astronomical Journal, 112, 1709–1729.
  • FRYZLEWICZ, P., VON SACHS, R. and VAN BELLEGEM, S. (2003). Forecasting non-stationary time series by wavelet process modelling. Ann. Inst. Math. Statist., 55, 737–764.
  • GANNAZ, I. (2006). Robust estimation and wavelet thresholding in partial linear models. Technical report, University Joseph Fourier, Grenoble, France. URL, http://www.citebase.org/abstract?id=oai:arXiv.org:math/0612066.
  • GAO, H.-Y. (1998). Wavelet shrinkage denoising using the non-negative garrote. J. Comput. Graph. Statist., 7, 469–488.
  • GAO, H.-Y. and BRUCE, A. (1997). Waveshrink with firm shrinkage. Statist. Sinica, 7, 855–874.
  • GREEN, P. and YANDELL, B. (1985). Semi-parametric generalized linear models. Technical Report 2847, University of, Wisconsin-Madison.
  • HALL, P., KERKYACHARIAN, G. and PICARD, D. (1998). Block threshold rules for curve estimation using kernel and wavelet methods. Ann. Statist., 26, 922–942.
  • HALL, P., KERKYACHARIAN, G. and PICARD, D. (1999). On the minimax optimality of block thresholded wavelet estimators. Statist. Sinica, 9, 33–50.
  • HALL, P., PENEV, S., KERKYACHARIAN, G. and PICARD, D. (1997). Numerical performance of block thresholded wavelet estimators. Statistics and Computing, 7, 115–124.
  • HALL, P. and TURLACH, B. (1997). Interpolation methods for nonlinear wavelet regression with irregularly spaced design. Ann. Statist., 25, 1912–1925.
  • HAMILTON, S. and TRUONG, Y. (1997). Local estimation in partly linear models. J. Multivariate Analysis, 60, 1–19.
  • JANSEN, M., MALFAIT, M. and BULTHEEL, A. (1997). Generalized cross validation for wavelet thresholding. Signal Processing, 56, 33–44.
  • JOHNSTONE, I. (1994). Minimax bayes, asymptotic minimax and sparse wavelet priors. In Statistical Decision Theory and Related Topics, V (S. Gupta and J. Berger, eds.). Springer-Verlag, New York, 303–326.
  • KERKYACHARIAN, G. and PICARD, D. (2004). Regression in random design and warped wavelets. Bernoulli, 10, 1053–1105.
  • KOVAC, A. and SILVERMAN, B. W. (2000). Extending the scope of wavelet regression methods by coefficient-dependent thresholding. J. Amer. Statist. Assoc., 95, 172–183.
  • LADA, E. K., LU, J.-C. and WILSON, J. R. (2002). A wavelet based procedure for process fault detection. IEEE Transactions on Semiconductor Manufacturing, 15, 79–90.
  • LORENZ, D. (2006). Non-convex variational denoising of images: Interpolation between hard and soft wavelet shrinkage. Current Development in Theory and Applications of Wavelets to, appear.
  • MALLAT, S. (1999). A Wavelet Tour of Signal Processing. 2nd ed. Academic Press, San, Diego.
  • MARRON, J., ADAK, S., JOHNSTONE, I., NEUMANN, M. and PATIL, P. (1998). Exact risk analysis of wavelet regression. J. Comput. Graph. Statist., 7, 278–309.
  • MARTENS, H. and NAES, T. (1989). Multivariate Calibration. John Wiley & Sons, New, York.
  • MAXIM, V. (2003). Restauration de signaux bruités observés sur des plans d’expérience aléatoires. Ph.D. thesis, University Joseph Fourier, France.
  • MEYER, Y. (1992). Wavelets and Operators. Cambridge University Press, Cambridge.
  • MORRIS, J. S. and CARROLL, R. J. (2006). Wavelet-based functional mixed models. J. Roy. Statist. Soc. Ser. B, 68, 179–199.
  • MRÁZEK, P., WEICKERT, J. and STEIDL, G. (2003). Correspondences between wavelet shrinkage and nonlinear diffusion. In Scale Space Methods in Computer Vision (L. D. Griffin and M. Lillholm, eds.), vol. 2695 of Lecture Notes in Computer Science. Springer, Berlin, 101–116.
  • MÜLLER, P. and VIDAKOVIC, B. (1999). Bayesian inference in wavelet based models. In Lect. Notes Statist., vol. 141. Springer-Verlag, New, York.
  • NASON, G. (1996). Wavelet shrinkage using cross-validation. J. Roy. Statist. Soc. Ser. B, 58, 463–479.
  • NASON, G. and SILVERMAN, B. (1995). The stationary wavelet transform and some statistical applications. In Wavelets and Statistics (A. Antoniadis and G. Oppenheim, eds.), vol. 103 of Lect. Notes Statist. Springer-Verlag, New York, 281–300.
  • OGDEN, R. (1997). Essential Wavelets for Statistical Applications and Data Analysis. Birkhäuser, Boston.
  • OH, H.-S. and LEE, T. C. M. (2005). Hybrid local polynomial wavelet shrinkage: wavelet regression with automatic boundary adjustment. Comput. Statist. Data Anal., 48, 809–819.
  • PERCIVAL, D. and WALDEN, A. (2000). Wavelet Methods for Time Series Analysis. Cambridge University Press, Cambridge.
  • PERONA, P. and MALIK, J. (1990). Scale space and edge detection using anisotropic diffusion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12, 629–639.
  • RAMSAY, J. B. and LAMPART, C. (1998). The decomposition of economic relationships by time scale using wavelets: expenditure and income. Studies in NonLinear Dynamics and Econometrics, 3, 23–42.
  • RICE, J. (1986). Convergence rates for partially splined models. Statist. Probab. Lett., 4, 203–208.
  • SOLO, V. (2001). Invited discussion on “Regularization of wavelets approximations” by A. Antoniadis and J. Fan. J. Amer. Statist. Assoc., 96, 963–965.
  • SPECKMAN, P. (1988). Kernel smoothing in partial linear models. J. Roy. Statist. Soc. Ser. B, 50, 413–436.
  • STEIDL, G. and WEICKERT, J. (2002). Relations between soft wavelet shrinkage and total variation denoising. In Pattern Recognition (V. G. L., ed.), vol. 2449 of Lecture Notes in Computer Science. Springer, Berlin, 198–205.
  • STRANG, G. and NGUYEN, T. (1996). Wavelets and Filter Banks. Wellesley-Cambridge, Press.
  • VIDAKOVIC, B. (1998b). Wavelet based nonparametric bayes methods. In Practical Nonparametric and Semiparametric Bayesian Statistics (D. Dey, P. Müller and D. Sinha, eds.), vol. 133 of Lect. Notes Statist. Springer-Verlag, New York, 133–155.
  • VIDAKOVIC, B. (1999). Statistical Modeling by Wavelets. John Wiley & Sons, New, York.
  • WAHBA, G. (1990). Spline Models for Observational Data. SIAM, Philadelphia.
  • WEAVER, J. B., XU, J., M., H. D. and CROMWELL, L. D. (1991). Filtering noise from images with wavelet transforms. Magnetic Resonance in Medicine, 21, 288–295.
  • WEICKERT, J. (1998). Anisotropic Diffusion in Image Processing. Teubner, Stuttgart.
  • XIA, Y., TONG, H., LI, W. K. and ZHU, L.-X. (2002). An adaptive estimation of dimension reduction space. J. Roy. Statist. Soc. Ser. B, 64, 363–410.
  • YAO, Q. and TONG, H. (1994). On subset selection in nonparametric stochastic regression. Statist. Sinica, 4, 51–70.