{"title":"A Kernel Based Rejection Method for Supervised Classification","authors":"Abdenour Bounsiar, Edith Grall, Pierre Beauseroy","volume":12,"journal":"International Journal of Computer and Information Engineering","pagesStart":3907,"pagesEnd":3917,"ISSN":"1307-6892","URL":"https:\/\/publications.waset.org\/pdf\/14477","abstract":"In this paper we are interested in classification problems\r\nwith a performance constraint on error probability. In such\r\nproblems if the constraint cannot be satisfied, then a rejection option\r\nis introduced. For binary labelled classification, a number of SVM\r\nbased methods with rejection option have been proposed over the\r\npast few years. All of these methods use two thresholds on the SVM\r\noutput. However, in previous works, we have shown on synthetic data\r\nthat using thresholds on the output of the optimal SVM may lead to\r\npoor results for classification tasks with performance constraint. In\r\nthis paper a new method for supervised classification with rejection\r\noption is proposed. It consists in two different classifiers jointly\r\noptimized to minimize the rejection probability subject to a given\r\nconstraint on error rate. This method uses a new kernel based linear\r\nlearning machine that we have recently presented. This learning\r\nmachine is characterized by its simplicity and high training speed\r\nwhich makes the simultaneous optimization of the two classifiers\r\ncomputationally reasonable. The proposed classification method with\r\nrejection option is compared to a SVM based rejection method\r\nproposed in recent literature. Experiments show the superiority of\r\nthe proposed method.","references":"[1] A. Bounsiar, P. Beauseroy, and E. Grall, \"A straightforward svm\r\napproach for classification with constraints,\" in proceedings of EUSIPCO-\r\n05, Antalya, Turkey, September 2005.\r\n[2] E. Grall, P. Beauseroy, and A. Bounsiar, \"Classification avec contraintes\r\n: probl\u00e9matique et apprentissage d-une r\u00e8gle de d\u00e9cision par svm,\" in\r\nProceedings of GRETSI-05. 2005, Louvain-la-Naeuve, Belgium.\r\n[3] T. Ha, \"The optimum class-selective rejection rule,\" Transactions on\r\nPattern Analysis ans Machine Intelligence, vol. 19, no. 6, pp. 608-615,\r\n1997.\r\n[4] T. Horiuchi, \"Class-selective rejection rule to minimize the maximum\r\ndistance between selected classes,\" Pattern recognition, vol. 31, no. 10,\r\npp. 579-588, 1998.\r\n[5] E. Grall, P. Beauseroy, and A. Bounsiar, \"Multilabel classification rule\r\nwith performance constraints,\" To appear in proceedings of ICASSP-06.\r\nToulouse, France, May 14-19 2006.\r\n[6] C. K. Chow, \"An optimum character recognition system using decision\r\nfunctions,\" IEEE Trans. Electronic computers, vol. EC-6, pp. 247-254,\r\nDecember 1957.\r\n[7] C. K. Chow, \"On optimum error and reject trade-off,\" IEEE Transactions\r\non Information Theory, vol. 16, pp. 41-46, 1970.\r\n[8] G. Fumera, F. Roli, and G. Giacinto, \"Analysis of error-reject tradeoff\r\nin linearly combined multiple classifiers,\" Pattern Recognition, vol.\r\n33(12), pp. 2099-2101, 2000.\r\n[9] G. Fumera and F. Roli, \"Analysis of error-reject trade-off in linearly\r\ncombined multiple classifiers,\" Pattern Recognition, vol. 37(6), pp.\r\n1245-1265, 2004.\r\n[10] M. Golfarelli, D. Maio, and D. Maltoni, \"On the error-reject trade-off in\r\nbiometric verification systems,\" IEEE Transactions on Pattern analysis\r\nand Machine Intelligence, vol. 19(7), pp. 789-796, 1997.\r\n[11] L. K. Hansen, C. Lissberg, and P. Salomon, \"The error-reject tradeoff,\"\r\nOpen systems and Information Dynamics, vol. 4, pp. 159-185, 1997.\r\n[12] G. Fumera and F. Roli, \"Support vector machines with embedded\r\nrejection option,\" In: Lee S, Verri A (eds) Pattern recognition with\r\nsupport vector machines. Lecture notes in computer science, vol. 2388,\r\npp. 68-82, Springer, Berlin Heidelberg New York, 2002.\r\n[13] S. Mukherjee, P. Tamayo, D. Slonim, A. Verri, T. Golub, J.P. Mesirov,\r\nand T. Poggio, \"Support vector machine classification of microarray\r\ndata,\" AI Memo 1677, Massachusetts Institute of Technology, 1999.\r\n[14] J.C. Platt, \"Probabilistic outputs for support vector machines and comparisons\r\nto regularized likelihood methods,\" In: Smola AJ, Bartlet PL,\r\nSch\u00f6lkopf B, Schurmans D (eds) Advances in large margin classifiers,\r\npp. 61-74, MIT Press, 2004.\r\n[15] J.T. Kwok, \"Moderating the outputs of support vector machine classifiers,\"\r\nIEEE Transactions on Neural Networks, vol. 10, pp. 1018-1031,\r\n1999.\r\n[16] F. Tortorella, \"Reducing the classifcation cost of support vector classifiers\r\nthrough an roc-based reject rule,\" Pattern Anal. Applic., vol. 7, pp.\r\n128-143, 2004.\r\n[17] A. Bounsiar, E. Grall, and P. Beauseroy, \"Using svm for binary classification\r\nwith first type error constraint,\" in proceedings of ICSIT-05,\r\nAlgiers, Algeria, july 2005, pp. 494-499.\r\n[18] A. Bounsiar, P. Beauseroy, and E. Grall, \"Fast training and efficient\r\nlinear learning machine,\" To appear in proceedings of ICASSP-06.\r\nToulouse, France, May 14-19 2006.\r\n[19] V. Vapnik, The Nature of Statistical Learning Theory, Spring Verlag,\r\n1995.\r\n[20] V. Vapnik, Statistical Learning Theory, Wiley, 1998.\r\n[21] N. Cristianini and J. Shawe-Taylor, Support vector machines and other\r\nkernel-based learning methods, Combridge University Press, 2000.\r\n[22] C. J. C. Burges, \"A tutorial on support vector machines for pattern\r\nrecognition,\" Data Mining and Knowledge Discovery, vol. 2, no. 2, pp.\r\n121-167, 1998.\r\n[23] C. Cortes and V. Vapnik, \"Support vector networks,\" Machine Learning,\r\nvol. 20, pp. 273-279, 1995.\r\n[24] B. Sch\u00f6lkopf and A. J. Smola, Learning with kernels, MIT Press, MA,\r\n2002.\r\n[25] C. J Van Rijsbergen, Information retrieval, Butterworths, London, 1979.\r\n[26] J. A. Swets, Information Retrieval Systems, Bolt, Beranek and Newman.\r\nCambridge, Massachusetts, 1967.\r\n[27] C. Drummond and R. Holte, \"What roc curves can-t do (and cost curves\r\ncan),\" in Proceedings of the ROC Analysis in Artificial Intelligence, 1st\r\nInternational Workshop. 22 ao\u251c\u2557t 2004, pp. 19-26, Valencia, Espagne.\r\n[28] K. Fukunaga, Introduction to Statistical Patern Rcognition, Academic\r\nPress, New York, 2nd edition, 1990.","publisher":"World Academy of Science, Engineering and Technology","index":"Open Science Index 12, 2007"}