Simple generate-evaluate strategy for tight-budget parameter tuning problems
Ivars Dzalbs, Tatiana Kalganova · 2020
Good hyperparameter selection is essential for metaheuristic algorithm performance. Tuning is usually a time-consuming and tedious task that requires user expertise for the best results. Automated tuning algorithms can help to speed up this process and even lead to better parameter configurations; however, it requires a vast amount of computing resources. This is especially true for complex real-world problems where a single evaluation of configuration can take minutes, hours or even days. To overcome the problem, the eTuner and eTunerAlgo have been proposed. The distinctive feature of eTunerAlgo is that both algorithm selection and parameter tuning is done automatically. We evaluate proposed algorithms using three metaheuristics-Ant Colony optimization (ACO), Evolutionary Strategy (ES) and Imperialist Competitive Algorithm (ICA) and two NP-hard problems - Aerial Surveying Problem (ASP) and Multiple Knapsack Problem (MKP). Furthermore, a metaheuristic tuning benchmark containing 18,760 configurations is generated for efficient method evaluation. The experimental results show that our proposed approach is best suited for a low tuning budget, where within a given time only a small size of configurations can be evaluated.