Journal of Applied Mathematics

A Semisupervised Feature Selection with Support Vector Machine

Kun Dai, Hong-Yi Yu, and Qing Li

Full-text: Open access


Feature selection has proved to be a beneficial tool in learning problems with the main advantages of interpretation and generalization. Most existing feature selection methods do not achieve optimal classification performance, since they neglect the correlations among highly correlated features which all contribute to classification. In this paper, a novel semisupervised feature selection algorithm based on support vector machine (SVM) is proposed, termed SENFS. In order to solve SENFS, an efficient algorithm based on the alternating direction method of multipliers is then developed. One advantage of SENFS is that it encourages highly correlated features to be selected or removed together. Experimental results demonstrate the effectiveness of our feature selection method on simulation data and benchmark data sets.

Article information

J. Appl. Math., Volume 2013 (2013), Article ID 416320, 11 pages.

First available in Project Euclid: 14 March 2014

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier


Dai, Kun; Yu, Hong-Yi; Li, Qing. A Semisupervised Feature Selection with Support Vector Machine. J. Appl. Math. 2013 (2013), Article ID 416320, 11 pages. doi:10.1155/2013/416320.

Export citation


  • Y. Hong, S. Kwong, Y. Chang, and Q. Ren, “Unsupervised feature selection using clustering ensembles and population based incremental learning algorithm,” Pattern Recognition, vol. 41, no. 9, pp. 2742–2756, 2008.
  • J. L. Rodgers and W. A. Nicewander, “Thirteen ways to look at the correlation coefficient,” The American Statistician, vol. 42, no. 1, pp. 59–66, 1988.
  • J. Lin, J. Ming, and D. Crookes, “Robust face recognition with partial occlusion, illumination variation and limited training data by optimal feature selection,” IET Computer Vision, vol. 5, no. 1, pp. 23–32, 2011.
  • T. M. Cover and J. A. Thomas, Elements of Information Theory, Wiley-Interscience, Hoboken, NJ, USA, 2nd edition, 2006.
  • L. Talavera, C. Nord, and J. Girona, “Dependency-Based Feature Selection for Clustering Symbolic Data,” Intelligent Data Analysis, vol. 4, no. 1, pp. 19–28, 2000.
  • H. Elghazel and A. Aussem, “Feature selection for unsupervised learning using random cluster ensembles,” in Proceedings of the 10th IEEE International Conference on Data Mining (ICDM '10), pp. 168–175, Sydney, Australia, December 2010.
  • O. Chapelle, B. Scholkopf, and A. Zien, Semi-Supervised Learning, MIT Press, Cambridge, Mass, USA, 2006.
  • M. Sugiyama, T. Idé, S. Nakajima, and J. Sese, “Semi-supervised local Fisher discriminant analysis for dimensionality reduction,” Machine Learning, vol. 78, no. 1-2, pp. 35–61, 2010.
  • F. Bellal, H. Elghazel, and A. Aussem, “A semi-supervised feature ranking method with ensemble learning,” Pattern Recognition Letters, vol. 33, no. 10, pp. 1426–1432, 2012.
  • Z. Zhao and H. Lu, “Semi-supervised feature selection via spectral analysis,” in Proceedings of the 7th SIAM International Conference on Data Mining, pp. 641–646, April 2007.
  • L. Wang, J. Zhu, and H. Zou, “The doubly regularized support vector machine,” Statistica Sinica, vol. 16, no. 2, pp. 589–615, 2006.
  • H. Zou and T. Hastie, “Regularization and variable selection via the elastic net,” Journal of the Royal Statistical Society B, vol. 67, no. 2, pp. 301–320, 2005.
  • Y. Guibo, C. Yifei, and X. Xiaohui, “Efficient variable selection in support vector machines via the alternating direction method of multipliers,” Journal of Machine Learning Research, vol. 15, pp. 832–840, 2011.
  • T. Joachims, “Transductive inference for text classification using support vector machines,” Proceedings of the 16th International Conference on Machine Learning (ICML '99), pp. 200–209, 1999.
  • O. Chapelle, V. Sindhwani, and S. S. Keerthi, “Optimization techniques for semi-supervised support vector machines,” Journal of Machine Learning Research, vol. 9, pp. 203–233, 2008.
  • C. A. Floudas, Nonlinear and Mixed-Integer Optimization: Fun-damentals and Applications, Oxford University Press, New York, NY, USA, 1995.
  • R. Collobert, F. Sinz, J. Weston, and L. Bottou, “Large scale transductive SVMs,” Journal of Machine Learning Research (JMLR), vol. 7, pp. 1687–1712, 2006.
  • R. T. Rockafellar, “A dual approach to solving nonlinear programming problems by unconstrained optimization,” Mathematical Programming, vol. 5, pp. 354–373, 1973.
  • C. Wu and X.-C. Tai, “Augmented lagrangian method, dual methods, and split bregman iteration for ROF, Vectorial TV, and high order models,” SIAM Journal on Imaging Sciences, vol. 3, no. 3, pp. 300–339, 2010.
  • T. Goldstein and S. Osher, “The split Bregman method for L1-regularized problems,” SIAM Journal on Imaging Sciences, vol. 2, no. 2, pp. 323–343, 2009.
  • Y. Saad, Iterative Methods For Sparse Linear Systems, Society for Industrial Mathematics, 2003.
  • G.-B. Ye and X. Xie, “Split Bregman method for large scale fused Lasso,” Computational Statistics and Data Analysis, vol. 55, no. 4, pp. 1552–1569, 2011.
  • J. B. Hiriart-Urruty and C. Lemarechal, Convex Analysis and Minimization Algorithms, Springer, Berlin, Germany, 1993.
  • S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein, “Distributed optimization and statistical learning via the alternating direction method of multipliers,” Foundations and Trends in Machine Learning, vol. 3, no. 1, pp. 1–122, 2010.