Flexible Kernel Selection in Multitask Support Vector Regression
Carlos Ruiz, Carlos M. Alaíz, Alejandro Catalina, José R. Dorronsoro · 2019
Multitask Learning (MTL) aims to solve several related problems at the same time exploiting the similarities between them. In particular, Support Vector Machines (SVMs) can be used for MTL, providing a model that is potentially more flexible than a classical SVM trained over the data of all the tasks, and which can use more information than a set of independent models trained over each one of the tasks. Nevertheless, a major drawback of these SVMs is the large number of hyperparameters if no simplifying assumptions are made, which prevents from using standard selection methods as grid or random searches. In particular, both the common kernel and the kernels associated to each one of the tasks have to be selected. In this paper, we propose an approach to choose these kernels based on Gaussian Processes (GPs), whose Bayesian perspective allows one to deal naturally with several parameters. In particular, a GP is trained for each task, and the resultant kernel parameters are transferred to the SVM-based MTL model. Several experiment in real-world datasets show empirically the usefulness of this approach and the advantages of the GP-based kernel selection method.