Competitive and Collaborative Learning Accelerates the Convergence of Deep Convolutional Neural Networks
Yanbin Dang, Yuliang Yang, Yueyun Chen, Mengyu Zhu, Dehui Yin · 2022
In the training of convolutional neural networks (CNNs), the layer-by-layer learning based on the backpropagation (BP) algorithm causes that in each round of weights update, the learning of the latter layer determines the learning of the former layer, while the former layer cannot directly affect the latter layer. This means that the flow of error information is unidirectional, causing non-cooperative learning between layers, thereby reducing the convergence speed of the networks. In this work, we propose a network structure that evaluates the relative contribution of each layer in the CNNs to the final output error. During training, it indirectly realizes the bidirectional flow of information between layers, achieving the purpose of cross-layer collaborative learning. Our algorithm also fuses features at different scales on the detection networks, which we call the flexible feature fusion network(FFN). On public datasets, we have conducted rich experiments. With the help of FFN, the convergence speed of the object detection model is greatly improved. Without pre-training weight initialization, the convergence speed of the model is approximately doubled.