Improving Kernel Methods through Complex Data Mapping
Hang Zhou, Fabio Tozeto Ramos, Eric W. Nettleton · 2010
This paper introduces a simple yet powerful data transformation strategy for kernel machines. Instead of adapting the parameters of the kernel function w.r.t. the given data (as in conventional methods), we adjust both the kernel hyper-parameters and the given data itself. Using this approach, the input data is transformed to be more representative of the assumptions encoded in the kernel function. A novel complex mapping is proposed to nonlinearly adjust the data. Optimization of the data transformation parameters is performed in two different manners. Firstly, the complex data mapping parameters and kernel hyper-parameters are selected separately, with the former guided by frequency metrics and the latter under the Bayesian framework. Next, the complex data mapping parameters and kernel hyper-parameters are optimized simultaneously in a Bayesian formulation by creating a new category of "integrated kernel" with the complex data mapping embedded. Experiments using Gaussian Process learning have shown that both methods improve the learning accuracy in either classification or regression tasks, with the complex mapping embedded kernel approach outperforming the separate complex mapping one.