NashNorm: A Novel Nash Equilibrium-Inspired Batch Normalization Method for Deep Neural Networks
Jincheng Zhang · Preprints.org · 2025
Batch Normalization has been widely used in various deep learning architectures to alleviate training instability and accelerate convergence. Despite this, traditional batch normalization mechanisms usually treat neurons as independent individuals and fail to effectively model the potential dependencies between them. This paper introduces the Nash equilibrium idea in game theory for the first time and designs a normalization mechanism based on strategic interaction, called "Nash BatchNorm". By constructing a strategic feedback mechanism between neurons, this method enables neurons to compete with each other as "strategic individuals" during the normalization process, thereby achieving a more stable equilibrium state. Experimental results show that this method achieves improvements in multiple indicators on the MLP architecture, and its versatility also indicates that it can be widely used in various neural network structures such as convolutional networks and Transformers.