Speeding up Common Hyperparameter Optimization Methods by a Two-Phase-Search
Alexander Wendt, Marco Wuschnig, Martin Lechner · IECON 2020 The 46th Annual Conference of the IEEE Industrial Electronics Society · 2020
Hyperparameter search concerns everybody who works with machine learning. We compare publicly available hyperparameter searches on four datasets. We develop metrics to measure the performance of hyperparameter searches across datasets of different sizes as well as machine learning algorithms. Further, we propose a method of speeding up the search by using subsets of data. Results show that random search performs well compared to Bayesian methods and that a combined search can speed up the search by a factor of 5.