The Annals of Statistics

Asymptotic equivalence and adaptive estimation for robust nonparametric regression

T. Tony Cai and Harrison H. Zhou

Full-text: Open access

Abstract

Asymptotic equivalence theory developed in the literature so far are only for bounded loss functions. This limits the potential applications of the theory because many commonly used loss functions in statistical inference are unbounded. In this paper we develop asymptotic equivalence results for robust nonparametric regression with unbounded loss functions. The results imply that all the Gaussian nonparametric regression procedures can be robustified in a unified way. A key step in our equivalence argument is to bin the data and then take the median of each bin.

The asymptotic equivalence results have significant practical implications. To illustrate the general principles of the equivalence argument we consider two important nonparametric inference problems: robust estimation of the regression function and the estimation of a quadratic functional. In both cases easily implementable procedures are constructed and are shown to enjoy simultaneously a high degree of robustness and adaptivity. Other problems such as construction of confidence sets and nonparametric hypothesis testing can be handled in a similar fashion.

Article information

Source
Ann. Statist. Volume 37, Number 6A (2009), 3204-3235.

Dates
First available in Project Euclid: 17 August 2009

Permanent link to this document
http://projecteuclid.org/euclid.aos/1250515385

Digital Object Identifier
doi:10.1214/08-AOS681

Mathematical Reviews number (MathSciNet)
MR2549558

Zentralblatt MATH identifier
1191.62070

Subjects
Primary: 62G08: Nonparametric regression
Secondary: 62G20: Asymptotic properties

Keywords
Adaptivity asymptotic equivalence James–Stein estimator moderate deviation nonparametric regression quantile coupling robust estimation unbounded loss function wavelets

Citation

Cai, T. Tony; Zhou, Harrison H. Asymptotic equivalence and adaptive estimation for robust nonparametric regression. Ann. Statist. 37 (2009), no. 6A, 3204--3235. doi:10.1214/08-AOS681. http://projecteuclid.org/euclid.aos/1250515385.


Export citation

References

  • Bickel, P. J. and Ritov, Y. (1988). Estimating integrated squared density derivatives: Sharp best order of convergence estimates. Sankhyā Ser. A 50 381–393.
  • Brown, L. D., Cai, T. T., Low, M. G. and Zhang, C. (2002). Asymptotic equivalence theory for nonparametric regression with random design. Ann. Statist. 30 688–707.
  • Brown, L. D., Cai, T. T. and Zhou, H. H. (2008). Robust nonparametric estimation via wavelet median regression. Ann. Statist. 36 2055–2084.
  • Brown, L. D., Carter, A. V., Low, M. G. and Zhang, C.-H. (2004). Equivalence theory for density estimation, Poisson processes and Gaussian white noise with drift. Ann. Statist. 32 2074–2097.
  • Brown L. D. and Low, M. G. (1996a). A constrained risk inequality with applications to nonparametric functional estimation. Ann. Statist. 24 2524–2535.
  • Brown, L. D. and Low, M. G. (1996b). Asymptotic equivalence of nonparametric regression and white noise. Ann. Statist. 24 2384–2398.
  • Brown, L. D., Wang, Y. and Zhao, L. (2003). Statistical equivalence at suitable frequencies of GARCH and stochastic volatility models with the corresponding diffusion model. Statist. Sinica 13 993–1013.
  • Cai, T. (1999). Adaptive wavelet estimation: A block thresholding and oracle inequality approach. Ann. Statist. 27 898–924.
  • Cai, T. and Low, M. (2005). Nonquadratic estimators of a quadratic functional. Ann. Statist. 33 2930–2956.
  • Cai, T. and Low, M. (2006a). Adaptive confidence balls. Ann. Statist. 34 202–228.
  • Cai, T. and Low, M. (2006b). Optimal adaptive estimation of a quadratic functional. Ann. Statist. 34 2298–2325.
  • Cai, T. and Wang, L. (2008). Adaptive variance function estimation in heteroscedastic nonparametric regression. Ann. Statist. 36 2025–2054.
  • Cai, T. T. and Zhou, H. H. (2008). Asymptotic equivalence and adaptive estimation for robust nonparametric regression. Technical report, Dept. Statistics, Univ. Pennsylvania.
  • Cai, T. T. and Zhou, H. H. (2009). A data-driven block thresholding approach to wavelet estimation. Ann. Statist. 87 569–595.
  • Daubechies, I. (1992). Ten Lectures on Wavelets. SIAM, Philadelphia.
  • Delattre, S. and Hoffmann, M. (2002). Asymptotic equivalence for a null recurrent diffusion. Bernoulli 8 139–174.
  • DeVore, R. and Popov, V. (1988). Interpolation of Besov spaces. Trans. Amer. Math. Soc. 305 397–414.
  • Donoho, D. L. and Johnstone, I. M. (1995). Adapting to unknown smoothness via wavelet shrinkage. J. Amer. Statist. Assoc. 90 1200–1224.
  • Donoho, D. L. and Johnstone, I. M. (1998). Minimax estimation via wavelet shrinkage. Ann. Statist. 26 879–921.
  • Donoho, D. L., Johnstone, I. M., Kerkyacharian, G. and Picard, D. (1995). Wavelet shrinkage: Asymptopia (with discussion). J. Roy. Statist. Soc. Ser. B 57 301–369.
  • Donoho, D. L. and Nussbaum, M. (1990). Minimax quadratic estimation of a quadratic functional. J. Complexity 6 290–323.
  • Dümbgen, L. (1998). New goodness-of-fit tests and their application to nonparametric confidence sets. Ann. Statist. 26 288–314.
  • Efromovich, S. Y. and Low, M. (1996). On optimal adaptive estimation of a quadratic functional. Ann. Statist. 24 1106–1125.
  • Fan, J. (1991). On the estimation of quadratic functionals. Ann. Statist. 19 1273–1294.
  • Genon-Catalot, V., Laredo, C. and Nussbaum, M. (2002). Asymptotic equivalence of estimating a Poisson intensity and a positive diffusion drift. Ann. Statist. 30 731–753.
  • Genovese, C. R. and Wasserman, L. (2005). Confidence sets for nonparametric wavelet regression. Ann. Statist. 33 698–729.
  • Grama, I. and Nussbaum, M. (1998). Asymptotic equivalence for nonparametric generalized linear models. Probab. Theory Related Fields 111 167–214.
  • Grama, I. and Nussbaum, M. (2002). Asymptotic equivalence for nonparametric regression. Math. Methods Statist. 11 1–36.
  • Golubev, G. K., Nussbaum, M. and Zhou, H. H. (2009). Asymptotic equivalence of spectral density estimation and Gaussian white noise. Ann. Statist. To appear.
  • Johnstone, I. M. (2002). Function estimation and Gaussian sequence models. Unpublished manuscript.
  • Johnstone, I. M. and Silverman, B. W. (2005). Empirical Bayes selection of wavelet thresholds. Ann. Statist. 33 1700–1752.
  • Klemelä, J. (2006). Sharp adaptive estimation of quadratic functionals. Probab. Theory Related Fields 134 539–564.
  • Komlós, J., Major, P. and Tusnády, G. (1975). An approximation of partial sums of independent rv’s and the sample df. I. Z. Wahrsch. Verw. Gebiete 32 111–131.
  • Korostelev, A. P. (1993). Exact asymptotically minimax estimator for nonparametric regression in uniform norm. Theory Probab. Appl. 38 775–782.
  • Laurent, B. and Massart, P. (2000). Adaptive estimation of a quadratic functional by model selection. Ann. Statist. 28 1302–1338.
  • Le Cam, L. (1964). Sufficiency and approximate sufficiency. Ann. Math. Statist. 35 1419–1455.
  • Le Cam, L. (1986). Asymptotic Methods in Statistical Decision Theory. Springer, New York.
  • Lepski, O. V. (1990). On a problem of adaptive estimation in white Gaussian noise. Theory Probab. Appl. 35 454–466.
  • Li, K.-C. (1989). Honest confidence regions for nonparametric regression. Ann. Statist. 17 1001–1008.
  • Meyer, Y. (1992). Wavelets and Operators. Cambridge Univ. Press, Cambridge.
  • Milstein, G. and Nussbaum, M. (1998). Diffusion approximation for nonparametric autoregression. Probab. Theory Related Fields 112 535–543.
  • Nussbaum, M. (1996). Asymptotic equivalence of density estimation and Gaussian white noise. Ann. Statist. 24 2399–2430.
  • Pinsker, M. S. (1980). Optimal filtering of square integrable signals in Gaussian white noise. Problems Inform. Transmission 120–133.
  • Spokoiny, V. G. (1998). Adaptive and spatially adaptive testing of nonparametric hypothesis. Math. Methods Statist. 7 245–273.
  • Strang, G. (1992). Wavelet and dilation equations: A brief introduction. SIAM Rev. 31 614–627.
  • Tsybakov, A. B. (2004). Introduction a l’estimation nonparamétrique. Springer, New York.
  • Triebel, H. (1992). Theory of Function Spaces II. Birkhäuser, Basel.
  • Wang, Y. Z. (2002). Asymptotic nonequivalence of GARCH models and diffusions. Ann. Statist. 30 754–783.
  • Zhang, C.-H. (2005). General empirical Bayes wavelet methods and exactly adaptive minimax estimation. Ann. Statist. 33 54–100.
  • Zhou, H. H. (2006). A note on quantile coupling inequalities and their applications. Available at http://www.stat.yale.edu/~hz68.