Performance analysis of stochastic gradient descent and adaptive moment estimation optimization algorithms for convolutional neural networks
Sita M. Yadav, Sandeep Chaware · 2024
The optimizers like SGD (stochastic gradient descent) and ADAM (adaptive moment estimation) play important role in loss reduction and accuracy enhancement. It loads the model with the model weights that have the lowest validation loss in order to make predictions. The chapter presented detailed comparison of SGD and ADAM. The results prove that performance of optimizer with proper hyperparameter tuning improves the detection accuracy in videos objects. In this chapter, the AlexNet and GoogleNet architectures are implemented on MSCOCO and Pascal Visual Object Classes (VOC) dataset. The performance of SGD and ADAM were compared. We have reached a speed of 68 frames per second and object detection accuracy ranges in between 78 percent and 92 percent. The results showed that the ADAM optimizer gives better results than SGD. To get better results from SGD, the hyper-tuning parameters, dropouts, batch normalization, and learning rates need to be trained precisely. With ADAM optimizer such hassle is removed. The obtained results indicated the importance of optimizer selection for convolutional neural network (CNN)-based models. The results have shown with ADAM optimizer the AlexNet and GoogleNet performances are improved.