Electronic Journal of Statistics
- Electron. J. Statist.
- Volume 2 (2008), 103-117.
Structured variable selection in support vector machines
Seongho Wu, Hui Zou, and Ming Yuan
Abstract
When applying the support vector machine (SVM) to high-dimensional classification problems, we often impose a sparse structure in the SVM to eliminate the influences of the irrelevant predictors. The lasso and other variable selection techniques have been successfully used in the SVM to perform automatic variable selection. In some problems, there is a natural hierarchical structure among the variables. Thus, in order to have an interpretable SVM classifier, it is important to respect the heredity principle when enforcing the sparsity in the SVM. Many variable selection methods, however, do not respect the heredity principle. In this paper we enforce both sparsity and the heredity principle in the SVM by using the so-called structured variable selection (SVS) framework originally proposed in [20]. We minimize the empirical hinge loss under a set of linear inequality constraints and a lasso-type penalty. The solution always obeys the desired heredity principle and enjoys sparsity. The new SVM classifier can be efficiently fitted, because the optimization problem is a linear program. Another contribution of this work is to present a nonparametric extension of the SVS framework, and we propose nonparametric heredity SVMs. Simulated and real data are used to illustrate the merits of the proposed method.
Article information
Source
Electron. J. Statist. Volume 2 (2008), 103-117.
Dates
First available in Project Euclid: 22 February 2008
Permanent link to this document
http://projecteuclid.org/euclid.ejs/1203692405
Digital Object Identifier
doi:10.1214/07-EJS125
Mathematical Reviews number (MathSciNet)
MR2386088
Zentralblatt MATH identifier
1320.62154
Subjects
Primary: 68T10: Pattern recognition, speech recognition {For cluster analysis, see 62H30}
Secondary: 62G05: Estimation
Keywords
Classification Heredity Nonparametric estimation Support vector machine Variable selection
Citation
Wu, Seongho; Zou, Hui; Yuan, Ming. Structured variable selection in support vector machines. Electron. J. Statist. 2 (2008), 103--117. doi:10.1214/07-EJS125. http://projecteuclid.org/euclid.ejs/1203692405.
