Effective Mutation and Recombination for Evolving Convolutional Networks
Binay Dahal, Justin Z. Zhan · 2020
The major part of the success that we have had in deep learning is attributed to the proper design of the architecture of the network model. With all the hardware resources and mathematical foundation at hand, it is the clever assembling of the pieces that define the performance of that deep learning model. Almost all of the research in this field till date use the models that are designed by a human researcher using their past experiences or applying the iterative process of adding components to the model to see which one performs the best. As we strive towards the general Artificial Intelligence, this process of manually tuning the network to make it work on a specific task does not add up well for the cause. We are concerned with automating the task of network architecture design where the learning algorithm tries to discover the best performing model on itself. To do this, we employ an evolving algorithm in addition to the gradient descent to evolve and train the model. Specifically, we use a form of genetic algorithm with mutation and recombination operators to constantly change the architecture, while the commonly used gradient descent is used as a learning algorithm for each of the genetically procreated models. We propose a series of mutation operators and a method of recombination called Highest Varying k-Features Recombination(HVk-FR) to evolve the CNN models. Results show using our method of recombination on top of mutation yields the best accuracy.