Hierarchical Ant Colony for Simultaneous Classifier Selection and Hyperparameter Optimization
Victor Oliveira Costa, Cesar Ramos Rodrigues · 2018
To simultaneously perform model selection and hyperparameter optimization without human intervention, a hierarchical problem by nature, is the aim of the expanding area of Automated Machine Learning (AutoML). Although many search methods have been applied to approach this problem, swarm intelligence algorithms, largely used in model selection tasks with low computational cost, have not yet been considered in a hierarchical manner. Therefore, this work proposes Hierarchical Ant Colony Model Builder (HACOMB), a modification of the Ant Colony Optimization for mixed-variables (ACOMV) algorithm that enables it to tackle the hierarchical search space of the AutoML problem for classification tasks. The proposed method was evaluated in the job of selecting SVM kernels and optimizing its hyperparameters. Results of HACOMBon a suite of benchmark datasets show that the method is much faster, while as accurate as or better when compared to a naive approach to the problem.