A Neuron Splitting Algorithm with Weight Conservation for Adaptive Neural Networks
Elena S. Abramova, Alexey Orlov · 2025
This paper presents the development and experimental evaluation of a neuron splitting algorithm based on weight sum preservation, designed to address overfitting in neural networks under limited data and computational constraints. The proposed method dynamically identifies neurons exhibiting high learnability and low elasticity – indicators of potential overfitting – and replaces them with two new neurons while preserving the total weight contribution of the original neuron. This ensures minimal disruption to previously acquired knowledge and supports stable learning in adaptive architectures. The neuron splitting algorithm is employed within the Neural Network Alternate Incremental Learning Algorithm, which integrates two distinct operational states – active learning and sleep phases – to enable fast adaptation to new data while preserving previously acquired knowledge through deep weight consolidation. Experimental results demonstrate that the neuron splitting approach significantly improves accuracy retention across sequential tasks compared to a baseline fixed-architecture model. The study confirms the effectiveness of dynamic structural adaptation in reducing catastrophic forgetting and improving generalization performance. These findings contribute to the development of more resilient and scalable machine learning systems capable of operating under realistic conditions.