An Enhanced Convolutional Neural Network with DropConnect and Ranger Optimizer
Jingrui Ye · 2024
This paper addresses the limitations of existing convolutional neural network (CNN) models by proposing an advanced CNN algorithm (MCNN-RS) that incorporates Leaky ReLU,Swish activation functions, Batch Normalization, and the Ranger optimizer. The algorithm features a 10-layer CNN architecture that utilizes Leaky ReLU to avoid the “dying ReLU“problem, ensuring stable and faster convergence. Swish activation is employed in select layers to enhance non-linear feature extraction, while Batch Normalization is integrated to accelerate training and improve generalization. DropConnect is used between the fully connected and output layers to mitigate overfitting. The Ranger optimizer, combining RAdam and Lookahead techniques, is applied to minimize focal loss, effectively addressing class imbalance. Performance analysis on the MNIST and HCL2000 datasets demonstrates that the proposed algorithm achieves superior recognition rates, outperforming three benchmark algorithms with a maximum average recognition rate of 99.21%. Additionally, on the HCL2000 test set, the MCNN-RS algorithm surpasses an extreme learning machine optimized by a support vector machine, with a 3.98% improvement in average recognition rate.