The Annals of Statistics

Reducing variance in univariate smoothing

Ming-Yen Cheng, Liang Peng, and Jyh-Shyang Wu

Full-text: Open access


A variance reduction technique in nonparametric smoothing is proposed: at each point of estimation, form a linear combination of a preliminary estimator evaluated at nearby points with the coefficients specified so that the asymptotic bias remains unchanged. The nearby points are chosen to maximize the variance reduction. We study in detail the case of univariate local linear regression. While the new estimator retains many advantages of the local linear estimator, it has appealing asymptotic relative efficiencies. Bandwidth selection rules are available by a simple constant factor adjustment of those for local linear estimation. A simulation study indicates that the finite sample relative efficiency often matches the asymptotic relative efficiency for moderate sample sizes. This technique is very general and has a wide range of applications.

Article information

Ann. Statist. Volume 35, Number 2 (2007), 522-542.

First available in Project Euclid: 5 July 2007

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G08: Nonparametric regression 62G05: Estimation
Secondary: 60G20: Generalized stochastic processes

Bandwidth coverage probability kernel local linear regression nonparametric smoothing variance reduction


Cheng, Ming-Yen; Peng, Liang; Wu, Jyh-Shyang. Reducing variance in univariate smoothing. Ann. Statist. 35 (2007), no. 2, 522--542. doi:10.1214/009053606000001398.

Export citation


  • Abramson, I. S. (1982). On bandwidth variation in kernel estimates---a square root law. Ann. Statist. 10 1217--1223.
  • Brockmann, M., Gasser, T. and Herrmann, E. (1993). Locally adaptive bandwidth choice for kernel regression estimators. J. Amer. Statist. Assoc. 88 1302--1309.
  • Cheng, M.-Y. and Hall, P. (2002). Error-dependent smoothing rules in local linear regression. Statist. Sinica 12 429--447.
  • Cheng, M.-Y. and Hall, P. (2003). Reducing variance in nonparametric surface estimation. J. Multivariate Anal. 86 375--397.
  • Cheng, M.-Y., Hall, P. and Titterington, D. M. (1997). On the shrinkage of local linear curve estimators. Statist. Comput. 7 11--17.
  • Cheng, M.-Y. and Peng, L. (2006). Simple and efficient improvements of multivariate local linear regression. J. Multivariate Anal. 97 1501--1524.
  • Choi, E. and Hall, P. (1998). On bias reduction in local linear smoothing. Biometrika 85 333--345.
  • Cleveland, W. S. (1979). Robust locally weighted regression and smoothing scatterplots. J. Amer. Statist. Assoc. 74 829--836.
  • Fan, J. (1993). Local linear regression smoothers and their minimax efficiencies. Ann. Statist. 21 196--216.
  • Fan, J. and Gijbels, I. (1995). Data-driven bandwidth selection in local polynomial fitting: Variable bandwidth and spatial adaptation. J. Roy. Statist. Soc. Ser. B 57 371--394.
  • Fan, J. and Gijbels, I. (1996). Local Polynomial Modelling and Its Applications. Chapman and Hall, London.
  • Fan, J. and Marron, J. S. (1994). Fast implementations of nonparametric curve estimators. J. Comput. Graph. Statist. 3 35--56.
  • Gasser, T., Kneip, A. and Köhler, W. (1991). A flexible and fast method for automatic smoothing. J. Amer. Statist. Assoc. 86 643--652.
  • Hall, P. (1992). The Bootstrap and Edgeworth Expansion. Springer, New York.
  • Hall, P. and Turlach, B. A. (1997). Interpolation methods for adapting to sparse design in nonparametric regression (with discussion). J. Amer. Statist. Assoc. 92 466--477.
  • Hastie, T. and Loader, C. (1993). Local regression: Automatic kernel carpentry (with discussion). Statist. Sci. 8 120--143.
  • Jones, M. C., Linton, O. and Nielsen, J. P. (1995). A simple bias reduction method for density estimation. Biometrika 82 327--338.
  • Kogure, A. (1998). Effective interpolations for kernel density estimators. J. Nonparametr. Statist. 9 165--195.
  • Loader, C. (1999). Local Regression and Likelihood. Springer, New York.
  • Ruppert, D. (1997). Empirical-bias bandwidths for local polynomial nonparametric regression and density estimation. J. Amer. Statist. Assoc. 92 1049--1062.
  • Ruppert, D., Sheather, S. J. and Wand, M. P. (1995). An effective bandwidth selector for local least squares regression. J. Amer. Statist. Assoc. 90 1257--1270.
  • Samiuddin, M. and El-Sayyad, G. M. (1990). On nonparametric kernel density estimates. Biometrika 77 865--874.
  • Seifert, B. and Gasser, T. (1996). Finite-sample variance of local polynomials: Analysis and solutions. J. Amer. Statist. Assoc. 91 267--275.
  • Seifert, B. and Gasser, T. (2000). Data adaptive ridging in local polynomial regression. J. Comput. Graph. Statist. 9 338--360.
  • Stone, C. J. (1977). Consistent nonparametric regression (with discussion). Ann. Statist. 5 595--645.
  • Tibshirani, R. and Hastie, T. (1987). Local likelihood estimation. J. Amer. Statist. Assoc. 82 559--567.
  • Wand, M. P. and Jones, M. C. (1995). Kernel Smoothing. Chapman and Hall, London.