ResFedAvg: Restricting Extreme Updates
Atul V Nirpase, Manisha J. Nene, Rajendra S. Deodhar · 2024
Federated machine learning is a distributed learning paradigm where multiple clients work together to train a common model without the need to share local data. FedAvg is the foundational Federated Learning (FL) algorithm that considers weights from all the participating client for arithmetic averaging to get a global model. RobFedAvg considers the extreme weights as outliers and prunes node-wise outliers before aggregation. Considering the high heterogeneity of datasets across the participating clients, the extreme weights represent significant learning on clients with highly heterogeneous data and shall not be outliers always. This paper proposes Restricted Federated Averaging (ResFedAvg), a new aggregation algorithm that considers extreme weights but limits its effect on averaging. Experimental evaluations are carried out on MNIST with IID and non-IID datasets. ResFedAvg performs better than RobFedAvg in terms of convergence and accuracy for IID and non-IID datasets.