Performance Investigations on Integrating Federated Learning with Future Networks
Shun Fukumoto, Ruidong Li, Zhou Su · 2023
For future networks, communications and computing will converge to provide services; Federated Learning (FL), as one of the typical distributed computing technologies, needs to be integrated with networking. For such integration, FL suffers from the straggler effect that the entire learning speed can be lowered down, because of the existence of the devices taking more time to complete their tasks. There are many existing works targeting at reducing straggler effects; However, they lacks the detailed investigations on the reasons and the impact of each cause when integrating FL with networking. To carefully investigate those aspects, we classify the reasons of such effects into 3 categories, computing power, communication capability and data distributions, and conduct the extensive experiments with carefully designs. After investigations, it is observed that learning completion time cannot be estimated by formulation with FLoating-point Operations Per second (FLOPs) if the device's computing capability is low. Also the communication time can be reduced by intentionally selecting appropriate devices when the computing powers of devices are heterogeneous, and the model parameters can be discarded if the device holds independent and identically distributed (i.i.d.) dataset.