Libra: A Fairness-Guaranteed Framework for Semi-Asynchronous Federated Learning
Chun Wang, Huawei Huang, Ruixin Li, Jialiang Liu, Ting Cai, Zibin Zheng · 2024
Federated Learning (FL) is a promising distributed machine learning framework that allows clients to collaboratively train a global model without data leakage. The synchronous FL suffers from the inefficient training caused by the slow-speed clients, which are called stragglers. Though asynchronous FL can well address the efficiency challenge, it induces massive system overheads and model degradation. As a framework considering the trade-off between synchronous and asynchronous FL, semi-asynchronous FL gains increasing attention. However, when clients' resources become a bottleneck, an unfair client scheduling may degrade global training accuracy and increase system overheads, especially in heterogeneous environments. In this paper, we propose Libra, which is a new FL framework aiming to achieve fair client scheduling in semi-asynchronous FL mode. Libra restricts devices that train too fast according to the model discrepancy. Furthermore, it selects stale local models according to the number of participating into FL training by clients. Additionally, Libra conducts a biased client selection while considering clients' resources and local losses. The experimental results show that Libra outperforms other baselines in terms of convergence accuracy, system overhead, and fairness of client participation. We also conduct an ablation study to further prove the effectiveness of Libra. In brief, Libra can achieve fair client scheduling and reduce inefficient local updates.