On the Support Vector Machine
Yi Lin · 2003
Classification is a fundamental problem at the intersection of machine learning and statistics. Machine learning methods have enjoyed considerable empirical success. However, they often have an ad hoc quality. It is desirable to have hard theoretical results which might highlight specific quantitative advantages of these methods. The statistical methods often tackle the classification problem through density estimation or regression. Theoretical properties of these statistical methods can be established, but only under the assumption of a fixed order of smoothness. Whether these methods work well when the assumptions are violated is not clear. The support vector machine (SVM) methodology is a rapidly growing area in machine learning, and is receiving considerable attention in recent years. The SVM has proved highly successful in a number of practical classification studies. In this paper we show that the SVM enjoys excellent theoretical properties which explain the good performance of the SVM. We show that the SVM approaches the the theoretical optimal classification rule (the Bayes rule) in a direct fashion, and its expected misclassification