Straggler-resilient Federated Learning: Tackling Computation Heterogeneity with Layer-wise Partial Model Training in Mobile Edge Network
Hongda Wu, Ping Wang, C V Aswartha Narayana · IEEE Transactions on Network Science and Engineering · 2025
Federated Learning (FL) enables many resource-limited devices to train a model collaboratively without data sharing. However, many existing works focus on model-homogeneous FL, where the global and local models are the same size, ignoring the inherently heterogeneous computational capabilities of different devices and restricting resource-constrained devices from contributing to FL. In this paper, we consider model-heterogeneous FL and propose Federated Partial Model Training (FedPMT), where devices with smaller computational capabilities work on partial models (subsets of the global model) and contribute to the global model. Different from Dropout-based partial model generations, which remove neurons in (hidden) model layers at random, model training inFedPMTis achieved from the back-propagation perspective. As such, all devices inFedPMTprioritize the most crucial parts of the global model. Theoretical analysis shows that the proposed partial model training design has a similar convergence rate to the widely adopted Federated Averaging (FedAvg) algorithm,$\mathcal {O}(1/T)$, with the sub-optimality gap enlarged by a constant factor related to the model splitting design inFedPMT. Empirical results show thatFedPMTsignificantly outperforms the existing partial model training designs, FedDrop and HeteroFL, especially on complex tasks. Meanwhile, compared to the popular model-homogeneous benchmark, FedAvg,FedPMTreaches the learning target in a shorter completion time, thus achieving a better trade-off between learning accuracy and completion time.