Open Access
Translator Disclaimer
2018 Robust boosting with truncated loss functions
Zhu Wang
Electron. J. Statist. 12(1): 599-650 (2018). DOI: 10.1214/18-EJS1404


Boosting is a powerful machine learning tool with attractive theoretical properties. In recent years, boosting algorithms have been extended to many statistical estimation problems. For data contaminated with outliers, however, development of boosting algorithms is very limited. In this paper, innovative robust boosting algorithms utilizing the majorization-minimization (MM) principle are developed for binary and multi-category classification problems. Based on truncated loss functions, the robust boosting algorithms share a unified framework for linear and nonlinear effects models. The proposed methods can reduce the heavy influence from a small number of outliers which could otherwise distort the results. In addition, adaptive boosting for the truncated loss functions are developed to construct more sparse predictive models. We present convergence guarantees for smooth surrogate loss functions with both iteration-varying and constant step-sizes. We conducted empirical studies using data from simulations, a pediatric database developed for the US Healthcare Cost and Utilization Project, and breast cancer gene expression data. Compared with non-robust boosting, robust boosting improves classification accuracy and variable selection.


Download Citation

Zhu Wang. "Robust boosting with truncated loss functions." Electron. J. Statist. 12 (1) 599 - 650, 2018.


Received: 1 August 2016; Published: 2018
First available in Project Euclid: 27 February 2018

zbMATH: 1386.68143
MathSciNet: MR3769190
Digital Object Identifier: 10.1214/18-EJS1404

Primary: 62G35 , 62H30
Secondary: 68Q32 , 90C26

Keywords: boosting , difference of convex , machine learning , MM algorithm , Robust method


Vol.12 • No. 1 • 2018
Back to Top