Comprehensive Analysis of the Impact of Learning Rate and Dropout Rate on the Performance of Convolutional Neural Networks on the CIFAR-10 Dataset
Changyu Peng · Applied and Computational Engineering · 2024
This study investigates the impact of two crucial hyperparameters, learning rate and dropout rate, on the performance of Convolutional Neural Networks (CNNs) on the CIFAR-10 dataset. Learning rate controls how quickly a model adapts to new data, while dropout rate helps prevent overfitting by randomly omitting neurons during training. These parameters are critical because they directly influence model convergence speed, accuracy, and generalization ability. By experimenting with various combinations of learning rates (0.01, 0.001, 0.0001) and dropout rates (0.3, 0.5, 0.7), we analyze the model's performance in terms of accuracy and overfitting. Our results suggest that a learning rate of 0.001 combined with a dropout rate of 0.5 yields the best balance between learning efficiency and generalization. This study offers important insights for optimizing hyperparameters in CNN training.