Electronic Journal of Statistics

Spatial adaptation in heteroscedastic regression: Propagation approach

Nora Serdyukova

Full-text: Open access

Abstract

The paper concerns the problem of pointwise adaptive estimation in regression when the noise is heteroscedastic and incorrectly known. The use of the local approximation method, which includes the local polynomial smoothing as a particular case, leads to a finite family of estimators corresponding to different degrees of smoothing. Data-driven choice of localization degree in this case can be understood as the problem of selection from this family. This task can be performed by a suggested in Katkovnik and Spokoiny (2008) FLL technique based on Lepski’s method. An important issue with this type of procedures – the choice of certain tuning parameters – was addressed in Spokoiny and Vial (2009). The authors called their approach to the parameter calibration “propagation”. In the present paper the propagation approach is developed and justified for the heteroscedastic case in presence of the noise misspecification. Our analysis shows that the adaptive procedure allows a misspecification of the covariance matrix with a relative error of order (logn)1, where n is the sample size.

Article information

Source
Electron. J. Statist., Volume 6 (2012), 861-907.

Dates
First available in Project Euclid: 21 May 2012

Permanent link to this document
https://projecteuclid.org/euclid.ejs/1337604768

Digital Object Identifier
doi:10.1214/12-EJS693

Mathematical Reviews number (MathSciNet)
MR2988432

Zentralblatt MATH identifier
1281.62090

Subjects
Primary: 62G05: Estimation
Secondary: 62G08: Nonparametric regression

Keywords
Adaptive estimation heteroscedastic data nonparametric regression Lepski’s method minimax rate of convergence model misspecification nonparametric regression oracle inequalities propagation

Citation

Serdyukova, Nora. Spatial adaptation in heteroscedastic regression: Propagation approach. Electron. J. Statist. 6 (2012), 861--907. doi:10.1214/12-EJS693. https://projecteuclid.org/euclid.ejs/1337604768


Export citation

References

  • [1] Akaike, H. (1973). Information theory and an extension of the maximum likelihood principle., Second International Symposium on Information Theory (Tsahkadsor, 1971), Akadémiai Kiadó, Budapest 267–281.
  • [2] Arlot, S. (2009). Model selection by resampling penalization., Electron. J. Stat. 3 557–624.
  • [3] Arlot, S. and Massart, P. (2009). Data-driven calibration of penalties for least squares regression., J. Mach. Learn. Res. 10(Feb) 245–279.
  • [4] Baraud, Y., Giraud, C. and Huet, S. (2009). Gaussian model selection with an unknown variance., Ann. Statist. 37:2 630–672.
  • [5] Birgé, L. and Massart, P. (2001). Gaussian model selection., Journal of the European Mathematical Society 3:3 203–268.
  • [6] Brillinger, D. R. (1977). Discussion of Stone (1977)., Ann. Statist. 5:4 622–623.
  • [7] Brua, J.-Y. (2009). Asymptotic efficient estimators for non-parametric heteroscedastic model., Statistical Methodology 6:1 47–60.
  • [8] Cleveland, W. S. (1979). Robust locally weighted regression and smoothing scatterplots., J. Amer. Statist. Assoc. 74:368 829–836.
  • [9] Dalalyan A. S. and Salmon J. (2011). Sharp Oracle Inequalities for Aggregation of Affine Estimators. Preprint, arXiv:1104.3969v3.
  • [10] Donoho, D. L. and Johnstone, I. M. (1994). Ideal spatial adaptation by wavelet shrinkage., Biometrica 81:3 425–455
  • [11] Fan, J. and Gijbels, I. (1996)., Local Polynomial Modelling and Its Applications. Monographs on Statistics and Applied Probability, 66. Chapman and Hall, London.
  • [12] Fan, J., Zhang, C. and Zhang, J. (2001). Generalized likelihood ratio statistics and Wilks phenomenon., Ann. Statist. 29:1 153–193.
  • [13] Galtchouk, L. and Pergamenshchikov, S. (2009). Adaptive asymptotically efficient estimation in heteroscedastic nonparametric regression, Journal of the Korean Statistical Society 38:4 305–322.
  • [14] Galtchouk, L. and Pergamenshchikov, S. (2009). Sharp non-asymptotic oracle inequalities for nonparametric heteroscedastic regression models., J. Nonparametr. Stat. 21:1 1–16.
  • [15] Goldenshluger, A. and Nemirovski, A. (1994). On spatial adaptive estimation of nonparametric regression., Research report, Technion-Israel Inst. Technology, Haifa, Israel.
  • [16] Golubev, Y. and Spokoiny, V. (2009). Exponential bounds for minimum contrast estimators., Electron. J. Stat. 3 712–746.
  • [17] Efromovich, S. and Pinsker, M. (1996). Sharp-optimal and adaptive estimation for heteroscedastic nonparametric regression., Statistica Sinica 6:4 925–942.
  • [18] Efromovich, S. (2007). Sequential design and estimation in heteroscedastic nonparametric regression., Sequential Analysis 26:1 3–25.
  • [19] Katkovnik, V. Ja. (1979). Linear and nonlinear methods of nonparametric regression analysis. (Russian), Soviet Automat. Control 5 35–46, 93.
  • [20] Katkovnik, V. Ja. (1983). Convergence of linear and nonlinear nonparametric estimates of “kernel” type., Automat. Remote Control 44:4 495–506; translated from Avtomat. i Telemekh. 1983 4 108–120 (Russian).
  • [21] Katkovnik, V. Ja. (1985)., Nonparametric Identification and Data Smoothing: The Method of Local Approximation. Nauka, Moscow (Russian).
  • [22] Katkovnik, V., Egiazarian, K. and Astola, J. (2006)., Local Approximation Techniques in Signal and Image Processing. Bellingham, WA: SPIE Press.
  • [23] Katkovnik, V. and Spokoiny, V. (2008). Spatially adaptive estimation via fitted local likelihood techniques., IEEE Trans. Signal Process., 56:3 873–886.
  • [24] Kerkyacharian, G., Lepski, O. and Picard, D. (2001). Nonlinear estimation in anisotropic multi-index denoising., Probab. Theory Related Fields 121:2 137–170.
  • [25] Kerkyacharian, G., Lepski, O. and Picard, D. (2007). Nonlinear estimation in anisotropic multiindex denoising. Sparse case., Teor. Veroyatn. Primen. 52:1 150–171; translation in Theory Probab. Appl. (2008). 52:1 58–77.
  • [26] Kullback, S. and Leibler, R. A. (1951). On information and sufficiency., Ann. Math. Statistics 22 79–86.
  • [27] Lepskii, O. V. (1990). A problem of adaptive estimation in Gaussian white noise. (Russian), Teor. Veroyatnost. i Primenen. 35:3 459–470; translation in Theory Probab. Appl. 35:3 454–466.
  • [28] Lepskii, O. V. (1992). Asymptotic minimax adaptive estimation. II. Schemes without optimal adaptation. Adaptive estimates. (Russian), Teor. Veroyatnost. i Primenen. 37:3 468–481; translation in Theory Probab. Appl. 37:3 433–448.
  • [29] Lepski, O. V., Mammen, E. and Spokoiny, V.G. (1997). Optimal spatial adaptation to inhomogeneous smoothness: an approach based on kernel estimates with variable bandwidth selectors., Ann. Stat. 25:3 929–947.
  • [30] Lepski, O. V. and Spokoiny, V.G. (1997). Optimal pointwise adaptive methods in nonparametric estimation., Ann. Stat. 25:6 2512–2546.
  • [31] Loader, C. (1999)., Local Regression and Likelihood. Statistics and Computing. Springer-Verlag, New York.
  • [32] Massart, P. (2003)., Concentration Inequalities and Model Selection (2007). Lectures from the 33rd Summer School on Probability Theory held in Saint-Flour, July 6–23, 2003. Lecture Notes in Mathematics, 1896. Springer Berlin.
  • [33] Ruppert, D. and Wand, M. P. (1994). Multivariate locally weighted least squares regression., Ann. Stat. 22:3 1346–1370.
  • [34] Saumard, A. (2010). Optimal upper and lower bounds for the true and empirical excess risks in heteroscedastic least-squares regression, Preprint hal-00512304, v1.
  • [35] Spokoiny, V. (2002). Variance estimation for high-dimensional regression models., J. Multivariate Anal. 82:1 111–133.
  • [36] Spokoiny, V. and Vial, C. (2009). Parameter tuning in pointwise adaptation using a propagation approach., Ann. Statist. 37:5B 2783–2807.
  • [37] Stone, C. J. (1977). Consistent nonparametric regression. With discussion and a reply by the author., Ann. Statist. 5:4 595–645.
  • [38] Stone, C. J. (1980). Optimal rates of convergence for nonparametric estimators., Ann. Statist. 8:6 1348–1360.
  • [39] Tibshirani, R. and Hastie, T. (1987). Local likelihood estimation., J. Amer. Statist. Assoc. 82:398 559–567.
  • [40] Tsybakov, A. B. (1986). Robust reconstruction of functions by a local approximation method. (Russian), Problemy Peredachi Informatsii 22:2 69–84 (Problems of Information Transmission, 1986 22:2 133–146).
  • [41] Tsybakov, A. B. (2009)., Introduction to Nonparametric Estimation. Springer Series in Statistics. Springer-Verlag, New York.
  • [42] White, H. (1982). Maximum likelihood estimation of misspecified models., Econometrica 50:1 1–25.