Statistical Science

Nonparametric Regressin with Correlated Errors

Jean Opsomer, Yuedong Wang, and Yuhong Yang

Full-text: Open access


Nonparametric regression techniques are often sensitive to the presence of correlation in the errors. The practical consequences of this sensitivity are explained, including the breakdown of several popular data-driven smoothing parameter selection methods. We review the existing literature in kernel regression, smoothing splines and wavelet regression under correlation, both for short-range and long-range dependence. Extensions to random design, higher dimensional models and adaptive estimation are discussed.

Article information

Statist. Sci., Volume 16, Issue 2 (2001), 134-153.

First available in Project Euclid: 24 December 2001

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Kernel regression splines wavelet regression adaptive estimation smoothing parameter selection


Opsomer, Jean; Wang, Yuedong; Yang, Yuhong. Nonparametric Regressin with Correlated Errors. Statist. Sci. 16 (2001), no. 2, 134--153. doi:10.1214/ss/1009213287.

Export citation


  • Adenstedt, R. K. (1974). On large sample estimation for the mean of a stationarysequence. Ann. Statist. 2 1095-1107.
  • Altman, N. (1994). Krige, smooth, both or neither? Technical report, Biometrics Unit, Cornell Univ.
  • Altman, N. S. (1990). Kernel smoothing of data with correlated errors. J. Amer. Statist. Assoc. 85 749-759.
  • Aronszajn, N. (1950). Theoryof reproducing kernels. Trans. Amer. Math. Soc. 68 337-404.
  • Barron, A. R. and Barron, R. L. (1988). Statistical learning networks: a unifying view. In Computer Science and Statistics: Proceedings of the 21st Interface, 192-203.
  • Beran, J. (1992). Statistical methods for data with long-range dependence. Statist. Sci. 7 404-416.
  • Beran, J. (1994). Statistics for Long-Memory Processes. Chapman and Hall, New York.
  • Bierens, H. (1983). Uniform consistencyof kernel estimators of a regression function under generalized conditions. J. Amer. Statist. Assoc. 78 699-707. Breiman, L., Friedman, J. H., Olshen, R. and Stone, C. J.
  • (1984). Classification and Regression Trees. Wadsworth, Belmont, CA.
  • Chiu, S.-T. (1989). Bandwidth selection for kernel estimate with correlated noise. Statist. Probab. Lett. 8 347-354.
  • Chu, C.-K. and Marron, J. S. (1991). Comparison of two bandwidth selectors with dependent errors. Ann. Statist. 19 1906-1918.
  • Collomb, G. and H¨ardle, W. (1986). Strong uniform convergence rates in robust nonparametric time series analysis. Stochastic Process. Appl. 23 77-89.
  • Cox, D. R. (1984). Long-range dependence: a review. In Statistics: An Appraisal. Proceedings 50th Anniversary Conference, (H. A. David and H. T. David, eds.) 55-74. Iowa State Univ. Press.
  • Diggle, P. J. and Hutchinson, M. F. (1989). On spline smoothing with autocorrelated errors. Austral. J. Statist. 31 166-182.
  • Donoho, D. L. and Johnstone, I. M. (1998). Minimax estimation via wavelet shrinkage. Ann. Statist. 26 879-921.
  • Efromovich, S. (1999). How to overcome curse of long-memory errors in nonparametric regression. IEEE Trans. Inform. Theory 45, 1735-1741.
  • Friedman, J. and Stuetzle, W. (1981). Projection pursuit regression. J. Amer. Statist. Assoc. 76 817-823.
  • Gu, C. and Wahba, G. (1993). Semiparametric ANOVA with tensor product thin plate spline. J. Roy. Statist. Assoc. Ser. B 55 353-368.
  • Hall, P. and Hart, J. D. (1990). Nonparametric regression with long-range dependence. Stochastic Process. Appl. 36 339-351.
  • Hall, P. Lahiri, S. N. and Polzehl, J. (1995). On bandwidth choice in nonparametric regression with both shortand long-range dependent errors. Ann. Statist. 23 1921-1936.
  • H¨ardle, W., Hall, P. and Marron, J. S. (1988). How far are automaticallychosen regression smoothing parameters from their optimum? J. Amer. Statist. Assoc. 83 86-95.
  • H¨ardle, W., Hall, P. and Marron, J. S. (1992). Regression smoothing parameters that are not far from their optimum. J. Amer. Statist. Assoc. 87 227-233.
  • Hart, D. (1997). Nonparametric Smoothing and Lack-of-Fit Tests. Springer, New York.
  • Hart, J. D. (1991). Kernel regression estimation with time series errors. J. Roy. Statist. Assoc. Ser. B 53 173-187.
  • Hart, J. D. (1994). Automated kernel smoothing of dependent data byusing time series cross-validation. J. Roy. Statist. Assoc. Ser. B 56 529-542.
  • Harville, D. (1976). Extension of the Gauss-Markov theorem to include the estimation of random effects. Ann. Statist. 4 384-395.
  • Herrmann, E., Gasser, T. and Kneip, A. (1992). Choice of bandwidth for kernel regression when residuals are correlated. Biometrika 79 783-795.
  • Johnstone, I. and Silverman, B. (1997). Wavelet threshold estimators for data with correlated noise. J. Roy. Statist. Assoc. Ser. B 59 319-351.
  • Kimeldorf, G. S. and Wahba, G. (1971). Some results on Tchebycheffian spline functions. J. Math. Anal. Appl. 33 82-94.
  • Kohn, R., Ansley, C. F. and Tharm, D. (1991). The performance of cross-validation and maximum likelihood estimators of spline smoothing parameters. J. Amer. Statist. Assoc. 86 1042-1050.
  • Kohn, R., Ansley, C. F. and Wong, C. (1992). Nonparametric spline regression with autoregressive moving average errors. Biometrika 79 335-346.
  • K ¨unsch, H., Beran, J. and Hampel, F. (1993). Contrasts under long-range correlations. Ann. Statist. 21 943-964.
  • Laslett, G. (1994). Kriging and splines: an empirical comparison of their predictive performance in some applications. J. Amer. Statist. Assoc. 89 392-409.
  • Nason, G. P. (1996). Wavelet shrinkage using cross-validation. J. Roy. Statist. Soc. Ser. B 58 463-479.
  • Nicoleris, T. and Yatracos, Y. G. (1997). Rate of convergence of estimates, Kolmogrov's entropyand the dimensionalityreduction principle in regression. Ann. Statist. 25 2493-2511.
  • Opsomer, J.-D. (1995). Estimating a function bylocal linear regression when the errors are correlated. Preprint 95-42, Dept. Statistics, Iowa State Univ.
  • Opsomer, J.-D. (1997). Nonparametric regression in the presence of correlated errors. In Modelling Longitudinal and Spatially Correlated Data: Methods, Applications and Future Directions (T. Gregoire, D. Brillinger, P. Diggle, E. RussekCohen, W. Warren and R. Wolfinger, eds.) 339-348. Springer, New York.
  • Opsomer, J.-D. and Ruppert, D. (1997). Fitting a bivariate additive model bylocal polynomial regression. Ann. Statist. 25 186-211.
  • Opsomer, J.-D. and Ruppert, D. (1998). A fullyautomated bandwidth selection method for fitting additive models by local polynomial regression. J. Amer. Statist. Assoc. 93 605-619.
  • Priestley, M. B. and Chao, M. T. (1972). Nonparametric function fitting. J. Roy. Statist. Assoc. Ser. B 34 385-392.
  • Ruppert, D., Sheather, S. J. and Wand, M. P. (1995). An effective bandwidth selector for local least squares regression. J. Amer. Statist. Assoc. 90 1257-1270.
  • Ruppert, D. and Wand, M. P. (1994). Multivariate locally weighted least squares regression. Ann. Statist. 22 1346-1370.
  • Stone, C. J. (1994). The use of polynomial splines and their tensor products in multivariate function estimation. Ann. Statist. 22 118-184. Stone, C. J. Hansen, M. H. Kooperberg, C. and Truong, Y.
  • (1997). Polynomial splines and their tensor products in extended linear modeling (with discussion). Ann. Statist. 25 1371-1470.
  • Wahba, G. (1978). Improper priors, spline smoothing, and the problem of guarding against model errors in regression. J. Roy. Statist. Assoc. Ser. B 40 364-372.
  • Wahba, G. (1983). Bayesian confidence intervals for the crossvalidated smoothing spline. J. Roy. Statist. Assoc. Ser. B 45 133-150.
  • Wahba, G. (1985). A comparison of GCV and GML for choosing the smoothing parameters in the generalized spline smoothing problem. Ann. Statist. 4 1378-1402.
  • Wahba, G. (1990). Spline Models for Observational Data. SIAM, Philadelphia.
  • Wahba, G. and Wang, Y. (1993). Behavior near zero of the distribution of GCV smoothing parameter estimates for splines. Statist. Probab. Lett. 25 105-111.
  • Wahba, G. Wang, Y., Gu, C., Klein, R. and Klein, B. (1995). Smoothing spline ANOVA for exponential families, with application to the Wisconsin Epidemiological Studyof Diabetic Retinopathy. Ann. Statist. 23 1865-1895.
  • Wang, Y. (1996). Function estimation via wavelet shrinkage for long-memorydata. Ann. Statist. 24 466-484. Wang, Y. (1998a). Mixed-effects smoothing spline ANOVA. J. Roy. Statist. Assoc. Ser. B 60 159-174. Wang, Y. (1998b). Smoothing spline models with correlated random errors. J. Amer. Statist. Assoc. 93 341-348.
  • Yang, Y. (1997). Nonparametric regression with dependent errors. Preprint 97-29, Dept. Statistics, Iowa State Univ. (A shorter version is accepted in Bernoulli.)
  • Yang, Y. (1999). Model selection for nonparametric regression. Statist. Sinica 9 475-499.
  • Yang, Y. (2000). Combining different procedures for adaptive regression. J. Multivariate Anal. 74 135-161.