Ridge Regression Learning Algorithm in Dual Variables
Craig J. Saunders, Alex Gammerman, Vladimir Vovk · ePrints Soton (University of Southampton) · 1998
In this paper we study a dual version of the Ridge Regression procedure. It allows us to perform non-linear regression by construct-ing a linear regression function in a high di-mensional feature space. The feature space representation can result in a large increase in the number of parameters used by the al-gorithm. In order to combat this “curse of dimensionality”, the algorithm allows the use of kernel functions, as used in Support Vector methods. We also discuss a powerful family of kernel functions which is constructed using the ANOVA decomposition method from the kernel corresponding to splines with an infi-nite number of nodes. This paper introduces a regression estimation algorithm which is a combination of these two elements: the dual version of Ridge Regression is applied to the ANOVA enhancement of the infinite-node splines. Experimental results are then presented (based on the Boston Housing data set) which indicate the performance of this algorithm relative to other algorithms. 1