Kernel matrix approximation for parameters tuning of support vector regression
Lizhong Ding, Shizhong Liao · 2010
Parameters tuning is fundamental for support vector regression (SVR). Previous tuning methods mainly adopted a nested two-layer optimization framework, where the inner one solved a standard SVR for fixed hyper-parameters and the outer one adjusted the hyper-parameters, which directly led to high computational complexity. To solve this problem, we propose a kernel matrix approximation algorithm KMA-α based on Monte Carlo and incomplete Cholesky factorization. The KMA-α approximates a given kernel matrix by a low-rank matrix, which will be used to feed SVR to improve its performance and further accelerate the whole parameters tuning process. Finally, on the basis of the computational complexity analysis of the KMA-α, we verify the performance improvement of parameters tuning attributed to the KMA-α on benchmark databases. Theoretical and experimental results show that the KMA-α is a valid and efficient kernel matrix approximation algorithm for parameters tuning of SVR.