Analysis of Pruned Deep Models Trained with Neuroevolution
Federico Da Rold, Léo Cazenille, Nathanaël Aubert-Kato · Proceedings of the Genetic and Evolutionary Computation Conference Companion · 2024
Deep neural networks show remarkable results in several fields of machine intelligence, such as effective classification in computer vision and robust adaptive systems in reinforcement learning scenarios. However, such models are typically overparametrized, leading to redundant neural pathways that massively increase the computational and energetic costs without providing any significant performance gain. Recent works have addressed the problem by proposing pruning techniques or using gradient-free neuroevolution to minimize model size. However, a systematic analysis of the effects of pruning in neural models during evolution is still missing. We propose an exploratory approach to fill this gap, relying on mathematical tools from network science to capture network structure regularities and information-theoretic analysis to describe the learning process. We focus on reinforcement learning problems solved by a quality diversity approach evolving a pruning operator is evolved along the models. Results from the analysis show the emergence of patterns and regimes in the mutual information and the estimation of network measures. This exploratory work will provide a solid ground for guiding and facilitating the development of pruning algorithms.