Communication-Efficient Resource Allocation for Wireless Federated Learning Systems
Chung-Hsuan Hu · Linköping studies in science and technology. Thesis · 2023
The training of machine learning (ML) models usually requires a massive amount of data.Nowadays, the ever-increasing number of connected user devices has benefited the development of ML algorithms by providing large sets of data that can be utilized for model training.As privacy concerns become vital in our society, using private data from user devices for training ML models becomes tricky.Therefore, federated learning (FL) with on-device information processing has been proposed for its advantages in preserving data privacy.FL is a collaborative ML framework where multiple devices participate in training a common global model based on locally available data.Unlike centralized ML architecture wherein the entire set of training data need to be centrally stored, in an FL system, only model parameters are shared between user devices and a parameter server.Federated Averaging (FedAvg) is one of the most representative and baseline FL algorithms, with an iterative process of model broadcasting, local training, and model aggregation.In every iteration, the model aggregation process can start only when all the devices have finished local training.Thus, the duration of one iteration is limited by the slowest device, which is known as the straggler issue.To resolve this commonly observed issue in synchronous FL methods, altering the synchronous procedure to an asynchronous one has been explored in the literature; that is, the server does not need to wait for all the devices to finish local training before conducting updates aggregation.However, to avoid high communication costs and implementation complexity that the existing asynchronous FL methods have brought in, we alternatively propose a new asynchronous FL framework with periodic aggregation.Since the FL process involves information exchanges over a wireless medium, allowing partial participation of devices in transmitting model updates is a common approach to avoid the communication bottleneck.We thus further develop channel-aware data-importance-based scheduling policies, which are theoretically motivated by the convergence analysis of the proposed FL system.In addition, an age-aware aggregation weighting design is proposed i