PEAR: privacy-preserving and effective aggregation for byzantine-robust federated learning in real-world scenarios
Han Sun, Yan Zhang, Huiping Zhuang, Jiatong Li, Zhi Xu, Liji Wu · The Computer Journal · 2024
Abstract Federated learning (FL) enables collaborative training of global models among distributed clients without sharing local data. Secure aggregation, a new security primitive of FL, enhances the confidentiality of data and model parameters. Unfortunately, privacy-preserving (PP) FL is vulnerable to common poisoning attacks by Byzantine adversaries. Existing defense strategies mainly focus on identifying abnormal local gradients over plaintexts, which provides a weak privacy guarantee. In PPFL, adversaries can escape existing defenses by uploading encrypted poisonous gradients. In addition, most mainstream aggregation algorithms assume that clients’ local training data is uniformly distributed, Independent and Identically Distributed (IID), which is unrealistic for real-world FL scenarios where data are only stored on large-scale terminal devices. To address these issues, we propose PEAR, a PP aggregation strategy based on single key-dual server CKKS full homomorphic encryption in real-world distributed scenarios, which can resist encrypted poisoning attacks. Specifically, we use cosine similarity to measure the distance between encrypted gradients. Then, we propose a novel Byzantine-tolerance aggregation mechanism using cosine similarity, which includes trust score generation that can tolerate differentiated local gradients and a two-step weight generation method that considers both the degree of gradient deviation in direction and training data size. This mechanism can achieve robustness for both IID and non-IID data without compromising privacy. Our extensive evaluations for two typical poisoning attacks on different datasets show that PEAR is robust and effective in IID and non-IID data and outperforms existing mainstream Byzantine-robust algorithms, especially achieving 16.4% to 53.2% testing error rate reduction in non-IID settings with significant label distribution and quantity skew while maintaining the same efficiency as FedAvg.