Faster optimization of SVR hyperparameters based on minimizing cross-validation error
Katsutaro Kobayashi, R. Nakano · 2005
The performance of support vector (SV) regression deeply depends on its hyperparameters such as the thickness of an insensitive zone, a penalty factor, kernel function parameters and so on. A method called MCV-SVR was recently proposed, which optimizes SVR hyperparameters lambda so that a cross-validation error is minimized. This paper proposes a faster version of the MCV-SVR. The MCV-SVR method iterates two basic steps until convergence; step 1 optimizes parameters thetas under given lambda, while step 2 improves lambda under given thetas. The present paper accelerates step 2 by effectively reducing the number of samples for evaluation. Our experiments using two data sets show that the CPU time for step 2 was reduced by more than one degree of magnitude and the total CPU time was reduced by half or more, while the generalization performance remained comparable