Bayesian Learning in Reproducing Kernel Hilbert Spaces: The Usefulness of the Bayes Point
Ralf Herbrich, Th Graepel, I C G Campbell · 1999
From a Bayesian perspective Support Vector Machines can be viewed as constructing a hypothesis using the center of the largest possible hypersphere which can be placed inside the version space, i.e. the space of all consistent hypotheses given a training set. The points of contact between this hypersphere and the boundaries of the version space define the support vectors. An alternative and potentially better approach is to construct the hypothesis using the whole version space. This can be achieved by using the Bayes point which is the midpoint of the region of intersection of all hyperplanes bisecting the version space into two volumes of equal size. It is known that the center of mass of the version space approximates the Bayes point [19]. Our approach to estimating the center of mass is to follow the trajectory of a billiard in the version space. We present experimental results which indicate that our algorithm consistently outperforms Support Vector Machines. 1 Introduction Recen...