L2 boosting in kernel regression
B.U. Park, Y.K. Lee, and S. Ha
Source: Bernoulli Volume 15, Number 3
(2009), 599-613.
Abstract
In this paper, we investigate the theoretical and empirical properties of L2 boosting with kernel regression estimates as weak learners. We show that each step of L2 boosting reduces the bias of the estimate by two orders of magnitude, while it does not deteriorate the order of the variance. We illustrate the theoretical findings by some simulated examples. Also, we demonstrate that L2 boosting is superior to the use of higher-order kernels, which is a well-known method of reducing the bias of the kernel estimate.
First Page:
Show
Hide
Full-text: Access denied (no subscription
detected)
We're sorry, but we are unable to provide
you with the full text of this article because we are not able to identify
you as a subscriber.
If you have a personal subscription to
this journal, then please login. If you are already logged in, then you
may need to update your profile to register your subscription. Read more about accessing full-text
Links and Identifiers
Permanent link to this document: http://projecteuclid.org/euclid.bj/1251463273
Digital Object Identifier: doi:10.3150/08-BEJ160
Mathematical Reviews number (MathSciNet): MR2555191
References
Bickel, P., Ritov, Y. and Zakai, A. (2006). Some theory for generalized boosting algorithms. J. Mach. Learn. Res. 7 705–732.
Mathematical Reviews (MathSciNet): MR2274384
Breiman, L. (1998). Arcing classifier. Ann. Statist. 26 801–824.
Mathematical Reviews (MathSciNet): MR1635406
Zentralblatt MATH: 0934.62064
Digital Object Identifier: doi:10.1214/aos/1024691079
Project Euclid: euclid.aos/1024691079
Breiman, L. (1999). Prediction games and arcing algorithms. Neural Computations 11 1493–1517.
Bühlmann, P. and Yu, B. (2003). Boosting with the L2 loss: Regression and classification. J. Amer. Statist. Assoc. 98 324–339.
Fan, J. and Gijbels, I. (1996). Local Polynomial Modelling and Its Applications. London: Chapman and Hall.
Mathematical Reviews (MathSciNet): MR1383587
Zentralblatt MATH: 0873.62037
Freund, Y. (1995). Boosting a weak learning algorithm by majority. Info. Comp. 121 256–285.
Mathematical Reviews (MathSciNet): MR1348530
Zentralblatt MATH: 0833.68109
Digital Object Identifier: doi:10.1006/inco.1995.1136
Freund, Y. and Schapire, R.E. (1996). Experiments with a new boosting algorithm. In Machine Learning: Proceedings of the Thirteenth International Conference 148–156.
Freund, Y. and Schapire, R.E. (1997). A decision-theoretic generalization of online learning and application to boosting. J. Comput. System Sci. 55 119–139.
Mathematical Reviews (MathSciNet): MR1473055
Zentralblatt MATH: 0880.68103
Digital Object Identifier: doi:10.1006/jcss.1997.1504
Friedman, J.H. (2001). Greedy function approximation: a gradient boosting machine. Ann. Statist. 29 1189–1232.
Mathematical Reviews (MathSciNet): MR1873328
Zentralblatt MATH: 1043.62034
Digital Object Identifier: doi:10.1214/aos/1013203451
Project Euclid: euclid.aos/1013203451
Friedman, J.H., Hastie, T and Tibshirani, R. (2000). Additive logistic regression: a statistical view of boosting. Ann. Statist. 28 337–407.
Mathematical Reviews (MathSciNet): MR1790002
Zentralblatt MATH: 1106.62323
Digital Object Identifier: doi:10.1214/aos/1016218223
Project Euclid: euclid.aos/1016218223
Györfi, L., Kohler, M., Krzyzak, A. and Walk, H. (2002). A Distribution-Free Theory of Nonparametric Regression. New York: Springer.
Mathematical Reviews (MathSciNet): MR1920390
Mason, L., Baxter, J., Bartlett, P. and Frean, M. (2000). Functional gradient techniques for combining hypotheses. In Advances in Large Margin Classifiers (A.J. Smola, P.J. Bartlett, B. Schölkopf and D. Schuurmans, eds.). Cambridge, MA: MIT Press.
Mathematical Reviews (MathSciNet): MR1820960
Ruppert, D., Sheather, S.J. and Wand, M.P. (1995). An effective bandwidth selector for local least squares regression. J. Amer. Statist. Assoc. 90 1257–1270.
Mathematical Reviews (MathSciNet): MR1379468
Zentralblatt MATH: 0868.62034
Digital Object Identifier: doi:10.2307/2291516
Schapire, R.E. (1990). The strength of weak learnability. Mach. Learn. 5 197–227.
Schapire, R.E., Freund, Y., Bartlett, P. and Lee, W.S. (1998). Boosting the margin: A new explanation for the effectiveness of voting methods. Ann. Statist. 26 1651–1686.
Mathematical Reviews (MathSciNet): MR1673273
Zentralblatt MATH: 0929.62069
Digital Object Identifier: doi:10.1214/aos/1024691352
Project Euclid: euclid.aos/1024691352
Schapire, R.E. and Singer, Y. (1999). Improved boosting algorithm using confidence-rated prediction. Mach. Learn. 37 297–336.
Mathematical Reviews (MathSciNet): MR1811573
Zentralblatt MATH: 0945.68194
Stützle, W. and Mittal, Y. (1979). Some comments on the asymptotic behavior of robust smoother. In Smoothing Techniques for Curve Estimation. Lecture Notes in Mathematics 757 191–195. Berlin: Springer.
Tukey, J.W. (1977). Exploratory Data Analysis. Reading, MA: Addison-Wesley.
2012 © Bernoulli Society for Mathematical Statistics and Probability
