Unveiling the Performance Dynamics and Stability of Artificial Neural Network Models in Reinforcement Learning: A Comprehensive Examination of ANN Size Influence
Martin Kaloev, Georgi Krastev · Research Square · 2023
Abstract This study assesses the performance and stability of artificial neural network (ANN) models in reinforcement learning (RL) simulations. Various simulations are conducted using multiple environments to examine the impact of ANN size on RL performance. Evaluation factors include rewards per episode, average reward stability, highest reward attainment, outliers analysis, action count, computation/training time. The findings reveal interesting patterns in RL model performance. Intermediate ANN sizes demonstrate optimal performance, indicating that neither excessively large nor small networks yield the best results. Smaller ANN models outperform larger ones in simulations with consistently high rewards for each action. Conversely, larger ANN models excel in simulations with sparse and infrequent rewards, benefiting from their stability. Small ANN models are advantageous in simulations with consistent action patterns, while larger models struggle to generalize effectively in low-reward situations. Supporting the hypothesis, the study confirms that using small ANN models with sufficient computing power to remember recurring patterns is superior to generalizing with larger ANN models. This research enhances our understanding of RL model dynamics and provides insights for selecting appropriate ANN sizes based on simulation characteristics.