Cooperative Training over Networks via Consensus-based Algorithms
Zhongguo Li, Bo Liu, Zhen Dong, Zhengtao Ding · 2021
In this paper, the training problem for a group of neural networks with private datasets is considered. Approximated gradients are employed to replace the true gradients in the proposed algorithms, due to the presence of gradient noises in the training problems. Consensus tools are used to achieve identical weights of the distributed neural networks trained using local dataset only. The convergence of the proposed algorithms is established by exploring the error dynamics of the connected agents, through which upper bounds for the learning rates are derived. Performances are analysed for the proposed algorithms with and without gradient noises. Simulation examples are provided to validate the effectiveness of the proposed algorithms.