Adaptive Local Update and Neural Composition for Accelerating Federated Learning in Heterogeneous Edge Networks
Jianchun Liu, Jiaming Yan, Ji Qi, Hongli Xu, Shilong Wang, Chunming Qiao, Liusheng Huang · IEEE Transactions on Networking · 2025
Federated Learning (FL) enables distributed clients to collaboratively train models without exposing their private data. However, it is difficult to implement efficient FL due to limited resources. Most existing works compress the transmitted gradients or prune the global model to reduce the resource cost, but leave the compressed or pruned parameters under-optimized, which degrades the training performance. To address this issue, the neural composition technique constructs size-adjustable models by composing low-rank tensors, allowing every parameter in the global model to learn the knowledge from all clients. Nevertheless, some tensors can only be optimized by a small fraction of clients, thus the global model may get insufficient training, leading to a long completion time, especially in heterogeneous edge scenarios. To this end, we enhance the neural composition technique, enabling all parameters to be fully trained. Further, we propose a lightweight FL framework, called Heroes, with enhanced neural composition and adaptive local update. A greedy-based algorithm is designed to adaptively assign the proper tensors and local update frequencies for participating clients according to their heterogeneous capabilities and resource budgets. On this basis, we further propose an extension of Heroes, termed AdaHeroes, which further improves the training performance under the statistical heterogeneity scenario based on an adaptive client selection strategy. Extensive experiments demonstrate that Heroes can reduce traffic consumption by about 72.46% and provide up to$2.76\times $speedup compared to the baselines. Furthermore, with the setting of statistical heterogeneity, AdaHeroes can improve the test accuracy by about 4.77% compared with Heroes and the baselines.