Fast Distributed Stochastic Nesterov Gradient Descent Algorithm for Image Classification
Dequan Li, Yuheng Zhang, Yuejin Zhou · 2021 China Automation Congress (CAC) · 2021
Based on stochastic gradient descent, this paper presents a fast distributed stochastic Nesterov gradient descent algorithm denoted by SFDGND, which can effectively solve the problem of image classification with distributed neural nets. The proposed SFDGND algorithm allows data to be randomly and evenly assigned to agents. Each agent updates its parameters with a local subset of data, thus enabling parallel computation. Finally, we compare the effectiveness of the proposed stochastic algorithm for training distributed neural nets optimizing problems. The experimental results are implemented on MNIST and CIFAR-10 datasets, which clearly demonstrates that SFDNGD algorithm performs well in practical applications and compares favorably to the existed algorithms.