Extended Bayesian framework for automatic tuning of kernel data-mining methods
Dmitry Kropotov, V. V. Ryazanov, Dmitry Vetrov · 2006
Kernel methods like e.g. Support vector machines (SVM) and Relevance vector machines (RVM) are widely used as data-mining tools. The concept of Bayesian learning exploited in RVMleads to Automatic relevance determination (ARD) which provides sparsity in resulting decision rules. This concept also sets all regularization coefficients without involving computationally expensive cross-validation methods. In this paper we suggest an extension of Bayesian maximal evidence framework which allows to set kernel function most appropriate for the particular task. We propose a local evidence estimation method which establishes a compromise between accuracy and stability of algorithm. In the paper we first briefly describe maximal evidence principle, present model of kernel algorithms as well as our approximations for evidence estimation, and then give results of experimental evaluation. Both classification and regression cases are considered.