Enhanced Federated Learning on Non-Iid Data via Local Importance Sampling
Zheqi Zhu, Pingyi Fan, Chenghui Peng, Khaled B. Letaief · 2023
As a privacy preserving framework, federated learning (FL) has been prevailing in numerous distributed scenarios. In this work, to enhance FL on non-iid data, we propose an explicit FL scheme with local importance sampling, Fed-Lis. Theoretically, we derive the convergence bound of Fed-Lis and obtain the design rule of the optimal sampling strategies. Experimentally, we formulate the algorithm and evaluate it on CIFAR-10. The results indicate that Fed-Lis reaps better accuracy, sampling efficiency, as well as robustness on non-iid data. Notably, as a non-iid FL solution from the local sampling aspect, Fed-Lis exhibits theoretical compatibility for non-convex loss functions. Furthermore, as a local sampling operation, Fed-Lis can be easily migrated into emerging FL frameworks.