Leveraged Vector Machines
Yoram Singer · 1999
We describe an iterative algorithm for building vector machines used in classification tasks. The algorithm builds on ideas from support vector machines, boosting, and generalized additive models. The algorithm can be used with various continuously differential functions that bound the discrete (0-1) classification loss and is very simple to implement. We test the proposed algorithm with two different loss functions on synthetic and natural data. We also describe a norm-penalized version of the algorithm for the exponential loss function used in AdaBoost. The performance of the algorithm on natural data is comparable to support vector machines while typically its running time is shorter than of SVM. 1 Introduction Support vector machines (SVM) [1, 13] and boosting [10, 3, 4, 11] are highly popular and effective methods for constructing linear classifiers. The theoretical basis for SVMs stems from Vapnik's seminal on learning and generalization [12] and has proved to be of gr...