Bernoulli

  • Bernoulli
  • Volume 20, Number 3 (2014), 1560-1599.

A robust, adaptive M-estimator for pointwise estimation in heteroscedastic regression

Michaël Chichignoud and Johannes Lederer

Full-text: Open access

Abstract

We introduce a robust and fully adaptive method for pointwise estimation in heteroscedastic regression. We allow for noise and design distributions that are unknown and fulfill very weak assumptions only. In particular, we do not impose moment conditions on the noise distribution. Moreover, we do not require a positive density for the design distribution. In a first step, we study the consistency of locally polynomial M-estimators that consist of a contrast and a kernel. Afterwards, minimax results are established over unidimensional Hölder spaces for degenerate design. We then choose the contrast and the kernel that minimize an empirical variance term and demonstrate that the corresponding M-estimator is adaptive with respect to the noise and design distributions and adaptive (Huber) minimax for contamination models. In a second step, we additionally choose a data-driven bandwidth via Lepski’s method. This leads to an M-estimator that is adaptive with respect to the noise and design distributions and, additionally, adaptive with respect to the smoothness of an isotropic, multivariate, locally polynomial target function. These results are also extended to anisotropic, locally constant target functions. Our data-driven approach provides, in particular, a level of robustness that adapts to the noise, contamination, and outliers.

Article information

Source
Bernoulli, Volume 20, Number 3 (2014), 1560-1599.

Dates
First available in Project Euclid: 11 June 2014

Permanent link to this document
https://projecteuclid.org/euclid.bj/1402488950

Digital Object Identifier
doi:10.3150/13-BEJ533

Mathematical Reviews number (MathSciNet)
MR3217454

Zentralblatt MATH identifier
1303.62035

Keywords
adaptation Huber contrast Lepski’s method M-estimation minimax estimation nonparametric regression pointwise estimation robust estimation

Citation

Chichignoud, Michaël; Lederer, Johannes. A robust, adaptive M-estimator for pointwise estimation in heteroscedastic regression. Bernoulli 20 (2014), no. 3, 1560--1599. doi:10.3150/13-BEJ533. https://projecteuclid.org/euclid.bj/1402488950


Export citation

References

  • [1] Antoniadis, A., Pensky, M. and Sapatinas, T. (2014). Nonparametric regression estimation based on spatially inhomogeneous data: Minimax global convergence rates and adaptivity. ESAIM Probab. Stat. 18 1–41.
  • [2] Arcones, M.A. (2005). Convergence of the optimal $M$-estimator over a parametric family of $M$-estimators. Test 14 281–315.
  • [3] Bertin, K. (2004). Estimation asymptotiquement exacte en norme sup de fonctions multidimensionnelles. Ph.D. thesis, Paris 6.
  • [4] Brown, L.D. and Low, M.G. (1996). A constrained risk inequality with applications to nonparametric functional estimation. Ann. Statist. 24 2524–2535.
  • [5] Cai, T.T. and Zhou, H.H. (2009). Asymptotic equivalence and adaptive estimation for robust nonparametric regression. Ann. Statist. 37 3204–3235.
  • [6] Chichignoud, M. (2012). Minimax and minimax adaptive estimation in multiplicative regression: Locally Bayesian approach. Probab. Theory Related Fields 153 543–586.
  • [7] Gaïffas, S. (2005). Convergence rates for pointwise curve estimation with a degenerate design. Math. Methods Statist. 14 1–27.
  • [8] Gaïffas, S. (2007). On pointwise adaptive curve estimation based on inhomogeneous data. ESAIM Probab. Stat. 11 344–364 (electronic).
  • [9] Gaïffas, S. (2007). Sharp estimation in sup norm with random design. Statist. Probab. Lett. 77 782–794.
  • [10] Giné, E. and Nickl, R. (2010). Confidence bands in density estimation. Ann. Statist. 38 1122–1170.
  • [11] Goldenshluger, A. and Lepski, O. (2008). Universal pointwise selection rule in multivariate function estimation. Bernoulli 14 1150–1190.
  • [12] Goldenshluger, A. and Nemirovski, A. (1997). On spatially adaptive estimation of nonparametric regression. Math. Methods Statist. 6 135–170.
  • [13] Hoffmann, M. and Nickl, R. (2011). On adaptive inference and confidence bands. Ann. Statist. 39 2383–2409.
  • [14] Huber, P.J. (1964). Robust estimation of a location parameter. Ann. Math. Statist. 35 73–101.
  • [15] Huber, P.J. (1981). Robust Statistics. Wiley Series in Probability and Mathematical Statistics. New York: Wiley.
  • [16] Huber, P.J. and Ronchetti, E.M. (2009). Robust Statistics, 2nd ed. Wiley Series in Probability and Statistics. Hoboken, NJ: Wiley.
  • [17] Katkovnik, V., Foi, A., Egiazarian, K. and Astola, J. (2010). From local kernel to nonlocal multiple-model image denoising. Int. J. Comput. Vis. 86 1–32.
  • [18] Katkovnik, V.Y. (1985). Neparametricheskaya Identifikatsiya i Sglazhivanie Dannykh. Metod Lokalnoi Approksimatsii. [The Method of Local Approximation]. Teoreticheskie Osnovy Tekhnicheskoĭ Kibernetiki. [Theoretical Foundations of Engineering Cybernetics]. Moscow: “Nauka”.
  • [19] Kerkyacharian, G., Lepski, O. and Picard, D. (2001). Nonlinear estimation in anisotropic multi-index denoising. Probab. Theory Related Fields 121 137–170.
  • [20] Klutchnikoff, N. (2005). Sur l’estimation adaptative de fonctions anisotropes. Ph.D. thesis, Aix-Marseille 1.
  • [21] Lambert-Lacroix, S. and Zwald, L. (2011). Robust regression through the Huber’s criterion and adaptive lasso penalty. Electron. J. Stat. 5 1015–1053.
  • [22] Lepski, O.V. and Levit, B.Y. (1999). Adaptive nonparametric estimation of smooth multivariate functions. Math. Methods Statist. 8 344–370.
  • [23] Lepski, O.V., Mammen, E. and Spokoiny, V.G. (1997). Optimal spatial adaptation to inhomogeneous smoothness: An approach based on kernel estimates with variable bandwidth selectors. Ann. Statist. 25 929–947.
  • [24] Lepskiĭ, O.V. (1990). A problem of adaptive estimation in Gaussian white noise. Teor. Veroyatn. Primen. 35 459–470.
  • [25] Massart, P. (2007). Concentration Inequalities and Model Selection. Lecture Notes in Math. 1896. Berlin: Springer. Lectures from the 33rd Summer School on Probability Theory held in Saint-Flour, July 6–23, 2003, with a foreword by Jean Picard.
  • [26] Polzehl, J. and Spokoiny, V. (2006). Propagation–separation approach for local likelihood estimation. Probab. Theory Related Fields 135 335–362.
  • [27] Reiss, M., Rozenholc, Y. and Cuenod, C. (2009). Pointwise adaptive estimation for robust and quantile regression. Available at arXiv:0904.0543v1.
  • [28] Spokoiny, V. and Vial, C. (2009). Parameter tuning in pointwise adaptation using a propagation approach. Ann. Statist. 37 2783–2807.
  • [29] Stone, C.J. (1975). Adaptive maximum likelihood estimators of a location parameter. Ann. Statist. 3 267–284.
  • [30] Tsybakov, A.B. (1982). Nonparametric signal estimation when there is incomplete information on the noise distribution. Probl. Inf. Transm. 18 44–60.
  • [31] Tsybakov, A.B. (1982). Robust estimates of a function. Problemy Peredachi Informatsii 18 39–52.
  • [32] Tsybakov, A.B. (1983). Convergence of nonparametric robust algorithms of reconstruction of functions. Avtomat. i Telemekh. 12 66–76.
  • [33] Tsybakov, A.B. (1986). Robust reconstruction of functions by a local approximation method. Problemy Peredachi Informatsii 22 69–84.
  • [34] Tsybakov, A.B. (2009). Introduction to Nonparametric Estimation. Springer Series in Statistics. New York: Springer. Revised and extended from the 2004 French original, translated by Vladimir Zaiats.
  • [35] van de Geer, S. and Lederer, J. (2013). The Bernstein–Orlicz norm and deviation inequalities. Probab. Theory Related Fields 157 225–250.