Surrogate-assisted neuroevolution
Bryson Greenwood, Tyler McDonnell · Proceedings of the Genetic and Evolutionary Computation Conference · 2022
Though Neuroevolution (NE) and Neural Architecture Search (NAS) have emerged as techniques for automating the design of neural networks, they are expensive and time consuming: they require training many neural networks and have largely resisted the benefits of surrogate-based optimization approaches, as it is difficult to model the performance of variable network architectures. We propose a novel and general framework for surrogate-assisted search of neural architectures consisting of two components: (1) an algorithm which leverages grammars to generate tensor representations of variable neural network topologies; and an evolutionary algorithm which employs a surrogate model to expedite architecture search using active learning. We demonstrate that our model can produce accurate performance predictions for unseen architectures, realizing a 5x reduction in the total compute required for search while improving asymptotic performance. We also illustrate that the surrogate models are transferable to new domains via a real-world transfer learning case study using industrial time series data.