The Annals of Statistics

Robust nonparametric estimation via wavelet median regression

Lawrence D. Brown, T. Tony Cai, and Harrison H. Zhou

Full-text: Open access

Abstract

In this paper we develop a nonparametric regression method that is simultaneously adaptive over a wide range of function classes for the regression function and robust over a large collection of error distributions, including those that are heavy-tailed, and may not even possess variances or means. Our approach is to first use local medians to turn the problem of nonparametric regression with unknown noise distribution into a standard Gaussian regression problem and then apply a wavelet block thresholding procedure to construct an estimator of the regression function. It is shown that the estimator simultaneously attains the optimal rate of convergence over a wide range of the Besov classes, without prior knowledge of the smoothness of the underlying functions or prior knowledge of the error distribution. The estimator also automatically adapts to the local smoothness of the underlying function, and attains the local adaptive minimax rate for estimating functions at a point.

A key technical result in our development is a quantile coupling theorem which gives a tight bound for the quantile coupling between the sample medians and a normal variable. This median coupling inequality may be of independent interest.

Article information

Source
Ann. Statist. Volume 36, Number 5 (2008), 2055-2084.

Dates
First available in Project Euclid: 13 October 2008

Permanent link to this document
http://projecteuclid.org/euclid.aos/1223908084

Digital Object Identifier
doi:10.1214/07-AOS513

Mathematical Reviews number (MathSciNet)
MR2458179

Zentralblatt MATH identifier
1148.62019

Subjects
Primary: 62G08: Nonparametric regression
Secondary: 62G20: Asymptotic properties

Keywords
Adaptivity asymptotic equivalence James–Stein estimator moderate large deviation nonparametric regression quantile coupling robust estimation wavelets

Citation

Brown, Lawrence D.; Cai, T. Tony; Zhou, Harrison H. Robust nonparametric estimation via wavelet median regression. Ann. Statist. 36 (2008), no. 5, 2055--2084. doi:10.1214/07-AOS513. http://projecteuclid.org/euclid.aos/1223908084.


Export citation

References

  • Antoniadis, A. and Fan, J. (2001). Regularization of wavelets approximations (with discussion). J. Amer. Statist. Assoc. 96 939–967.
  • Averkamp, R. and Houdré, C. (2003). Wavelet thresholding for non-necessarily Gaussian noise: Idealism. Ann. Statist. 31 110–151.
  • Averkamp, R. and Houdré, C. (2005). Wavelet thresholding for non-necessarily Gaussian noise: Functionality. Ann. Statist. 33 2164–2193.
  • Bretagnolle, J. and Massart, P. (1989). Hungarian constructions from the nonasymptotic view point. Ann. Probab. 17 239–256.
  • Brown, L. D., Cai, T. T., Zhang, R., Zhao, L. H. and Zhou, H. H. (2006). The Root-unroot algorithm for density estimation as implemented via wavelet block thresholding. Unpublished manuscript. Available at http://www-stat.wharton.upen.edu/~lbrown/.
  • Brown, L. D. and Low, M. G. (1996a). A constrained risk inequality with applications to nonparametric functional estimation. Ann. Statist. 24 2524–2535.
  • Brown, L. D. and Low, M. G. (1996b). Asymptotic equivalence of nonparametric regression and white noise. Ann. Statist. 24 2384–2398.
  • Burnashev, M. V. (1996). Asymptotic expansions for median estimate of a parameter. Theory Probab. Appl. 41 632–645.
  • Cai, T. (1999). Adaptive wavelet estimation: A block thresholding and oracle inequality approach. Ann. Statist. 27 898–924.
  • Cai, T. and Brown, L. D. (1998). Wavelet shrinkage for nonequispaced samples. Ann. Statist. 26 1783–1799.
  • Cai, T. T. and Zhou, H. H. (2006). A data-driven block thresholding approach to wavelet estimation. Available at http://www.stat.yale.edu/~hz68.
  • Casella, G. and Berger R. L. (2002). Statistical Inference. Duxbury Press, Pacific Grove, CA.
  • Csörgő, M. and Révész, P. (1978). Strong approximation of the quantile process. Ann. Statist. 6 882–894.
  • Daubechies, I. (1992). Ten Lectures on Wavelets. SIAM, Philadelphia.
  • DeVore and Popov (1988). Interpolation of Besov spaces. Trans. Amer. Math. Soc. 305 397–414.
  • Donoho, D. L. and Johnstone, I. (1994). Ideal spatial adaptation via wavelet shrinkage. Biometrika 81 425–455.
  • Donoho, D. L. and Johnstone, I. M. (1995). Adapting to unknown smoothness via wavelet shrinkage. J. Amer. Statist. Assoc. 90 1200–1224.
  • Donoho, D. L. and Johnstone, I. M. (1998). Minimax estimation via wavelet shrinkage. Ann. Statist. 26 879–921.
  • Donoho, D. L. and Yu, T. P.-Y. (2000). Nonlinear pyramid transforms based on median-interpolation. SIAM J. Math. Anal. 31 1030–1061.
  • Grama, I. and Nussbaum, M. (1998). Asymptotic equivalence for nonparametric generalized linear models. Probab. Theory Related Fields 111 167–214.
  • Hall, P. and Patil, P. (1996). On the choice of smoothing parameter, threshold and truncation in nonparametric regression by wavelet methods. J. Roy. Statist. Soc. Ser. B 58 361–377.
  • Komlós, J., Major, P. and Tusnády, G. (1975). An approximation of partial sums of independent rv’s and the sample df. I Z. Wahrsch. Verw. Gebiete 32 111–131.
  • Kovac, A. and Silverman, B. W. (2000). Extending the scope of wavelet regression methods by coefficient-dependent thresholding. J. Amer. Statist. Assoc. 95 172–183.
  • Lepski, O. V. (1990). On a problem of adaptive estimation in white Gaussian noise. Theor. Probab. Appl. 35 454–466.
  • Mason, D. M. (2001). Notes on the KMT Brownian bridge approximation to the uniform empirical process. In Asymptotic Methods in Probability and Statistics with Applications (N. Balakrishnan, I. A. Ibragimov and V. B. Nevzorov, eds.) 351–369. Birkhäuser, Boston.
  • Meyer, Y. (1992). Wavelets and Operators. Cambridge Univ. Press.
  • Nussbaum, M. (1996). Asymptotic equivalence of density estimation and Gaussian white noise. Ann. Statist. 24 2399–2430.
  • Pollard, D. P. (2001). A User’s Guide to Measure Theoretic Probability. Cambridge Univ. Press.
  • Stuck, B. W. and Kleiner, B. (1974). A statistical analysis of telephone noise. Bell System Technical J. 53 1263–1320.
  • Stuck, B. W. (2000). A historical overview of stable probability distributions in signal processing. IEEE International Conference on Acoustics, Speech, and Signal Processing 6 3795–3797.
  • Strang, G. (1992). Wavelet and dilation equations: A brief introduction. SIAM Rev. 31 614–627.
  • Triebel, H. (1992). Theory of Function Spaces. II. Birkhäuser, Basel.
  • Zhang, C.-H. (2005). General empirical Bayes wavelet methods and exactly adaptive minimax estimation. Ann. Statist. 33 54–100.
  • Zhou, H. H. (2005). A note on quantile coupling inequalities and their applications. Available at http://www.stat.yale.edu/~hz68.