Evolutionary Hyperparameter Optimisation for Sentence Classification
Brendan Rogers, Nasimul Noman, Stephan K. Chalup, Pablo Moscato · 2021
The performance that Deep Neural Networks can achieve on a specific task is impacted significantly by the hyperparameters selected. In order to compare the performance of various Deep Neural Network architectures on sentence classification, an optimised set of hyperparameters needs to be found for each architecture. In this work we use a simple Genetic Algorithm to optimise the hyperparameters of three different architectures and we evaluate their performance on a suite of sentence classification benchmarks. We found that a single Genetic Algorithm is capable of optimising a variety of different architectures and that the evolved configurations found can compete with those chosen by experts while using fewer overall trainable parameters. The three architectures tested are a recurrent neural network and two types of convolutional networks with a large difference in complexity. Of the three architectures, optimised for sentence classification, a simple Convolutional Neural Network was the overall best performer consistently achieving good performance while using very few trainable parameters.