Accelerating Wireless Distributed Learning through Hybrid Split and Federated Learning
Xuefei Li, Kun Guo, Xijun Wang, Ruifeng Gao, Howard Hua Yang · 2024
Federated learning (FL) and split learning (SL) are two prominent distributed learning modes. FL allows for parallel training but demands significant computational resources on devices to train deep neural network models. Conversely, SL reduces the computational burden on devices and can enhance learning performance, though it often leads to longer training time due to its sequential nature. In this paper, we introduce a novel distributed learning framework, hybrid split and federated learning (HSFL), which combines the advantages of both FL and SL over wireless networks. To achieve a lower training loss within a shorter latency, we start with the convergence analysis of HSFL, followed by a joint optimization problem of the learning mode selection, model splitting, and bandwidth allocation. To solve the problem, we propose a two-stage algorithm. First, we find the optimal bandwidth allocation and model splitting with a fixed learning mode. Then, we select the optimal learning mode based on the above optimal values. Experimental results validate the superior learning efficacy of our proposed algorithm.