Optimizing Support Vector regression hyperparameters based on cross-validation
K. Ito, Ryoko Nakano · 2004
This paper proposes a method to optimize hyperparameters for Support Vector (SV) regression so that the cross-validation error is minimized. The performance of SV regression depends on its hyperparameters such as /spl epsiv/ (the thickness of a tube), C (a penalty factor), /spl sigma/ (kernel function parameter), and so on. This paper employs the procedure of cross-validation to optimize these hyperparameters together with training the corresponding SV regression models; thus, the learning is performed by using a coordinate descent method. Since an error surface produced by the usual /spl epsiv/-insensitive l/sub 1/ loss is not smooth, not suitable for our approach, we introduce the /spl epsiv/-insensitive l/sub 2/ loss. The experiments show the l/sub 2/ loss produces very smooth error surfaces and our coordinate descent nicely works, reaching the model whose validation performance is globally optimal.