Optimal Margin Distribution Machine
Teng Zhang, Zhi‐Hua Zhou · IEEE Transactions on Knowledge and Data Engineering · 2019
Support Vector Machine (SVM) has always been one of the most successful learning algorithms, with the central idea of maximizing theminimum margin, i.e., the smallest distance from the instances to the classification boundary. However, recent theoretical results disclosed that maximizing the minimum margin does not necessarily lead to better generalization performance, and instead, themargin distributionhas been proven to be more crucial. Based on this idea, we propose the Optimal margin Distribution Machine (ODM), which can achieve a better generalization performance by optimizing the margin distribution explicitly. We characterize the margin distribution by the first- and second-order statistics, i.e., the margin mean and variance. The proposed method is a general learning approach which can be applied in any place where SVMs are used, and its superiority is verified both theoretically and empirically in this paper.