Decay Momentum for Improving Federated Learning
Miguel Fernandes, Catarina Silva, Joel Perdiz Arrais, Alberto Cardoso, Bernardete Ribeiro · ESANN 2021 proceedings · 2021
We propose two novel Federated Learning (FL) algorithms based on decaying momentum (Demon): Federated Demon (FedDemon) and Federated Demon Adam (FedDemonAdam).In particular, we apply Demon to Momentum Stochastic Gradient Descent (SGD) and Adam in a Federated setting, which has shown to improve results in a centralized environment.We empirically show that FedDemon and FedDemonAdam have a faster convergence rate and performance improvements compared to state-of-the-art algorithms including FedAvg, FedAvgM and FedAdam.17