FedBF16-Dynamic: Communication-Efficient Federated Learning with Adaptive Transmission
Fan‐Hsun Tseng, Yu-Hsiang Huang · 2024
Federated learning has a communication bottleneck since a considerable number of parameters are transferred between the central server and edge devices. Therefore, some prior works proposed compression methods by using the top-k sparsification to solve communication-efficient issue. However, we observe that this method affects the accuracy in the early stage of training. To address the problem, we propose a novel parameter upload mechanism, viz. FedBF16-Dynamic. In the early communication rounds of training, we apply the brain floating-point (Bfloat16) for transmission numerical type to upload model parameters, which reduces communication cost comprehensively. In subsequent communication rounds, there are two upload schemes for the edge devices with different levels of uplink bandwidth. Compared with the baseline, simulation results show that the proposed FedBF16-Dynamic scheme reduces communication cost and achieves higher performance within the least amount of time in various network environments.