A Novel Nash Equilibrium-Based Dynamic Connection Mechanism for Deep Neural Networks
Jincheng Zhang · Preprints.org · 2025
Skip Connections and Dense Connections are important structural designs widely used in deep learning models in recent years. They greatly alleviate the gradient vanishing problem in deep neural network training and improve the expressiveness and generalization performance of the model. This paper attempts to introduce the idea of Nash equilibrium theorem in game theory into this type of connection mechanism, and proposes a "Nash-Equilibrium Skip Connection". While keeping the structure simple, it establishes an "equilibrium state" information fusion method between multi-layer neuron outputs through an adaptive trade-off mechanism. Experimental results show that this method brings considerable performance improvement without increasing training time. This mechanism has good versatility and is not limited to the traditional multi-layer perceptron (MLP) model. It can be extended to various deep architectures such as CNN and Transformer.