Fed-SHARC: Resilient Decentralized Federated Learning based on Reward driven Clustering
Renuga Kanagavelu, Chris George Anil, Yuan Wang, Huazhu Fu, Qingsong Wei, Yong Liu, Rick Siow Mong Goh · 2024
Federated Learning (FL) is an attractive machine learning paradigm that facilitates collaborative machine learning at the decentralized level and produces insightful results. However, in cases where the participating clients have non-IID (non independent and identical data) distributions, it produces suboptimal results. Furthermore, gradient leakage attacks have been shown in federated learning to be capable of leaking confidential information from global model parameters, hence posing a severe risk to user privacy. In this work, we develop a method called Fed-SHARC, a Secure Hierarchical model Aggregation by clustering in a decentralized federated learning framework that addresses data heterogeneity issues to achieve good performance while guarding the framework against gradient leakage attacks. Secure model aggregation occurs at two stages in this hierarchical approach. Phase-1 involves exploiting differential privacy to aggregate the models of the reward-driven clustered clients. On the other hand, giving each participant with heterogeneous data the same privacy budget will have a significant impact on the performance. As an effective multi-participant budget allocation technique, we suggest an efficient weighted noise injection policy that modifies the privacy budget based on the data distributions of clients. Phase-2 involves selecting a leader from each cluster’s highly rewarded clients to take part in the multi-party computation enabled inter-cluster secure model aggregation. Experiments conducted on three benchmark public datasets demonstrate the effectiveness of the proposed Fed-SHARC in terms of privacy and performance. Furthermore, we verify its resilience against gradient leakage attacks.