Optimizing Client and Data Selection in Federated Learning: A Centralized Optimization and Decentralized Game-Theoretic Approach
Junkun Lin, Jingjing Luo, Tong Wang, Lin Xia Gao · 2023
Federated Learning (FL) is a distributed machine learning approach that enables multiple individual devices (clients) to collaboratively train a global machine learning model without directly sharing their raw data with each other. By keeping the raw data on local devices, FL can effectively preserve data privacy and security. However, the performance of FL system is highly dependent on the amount and quality of client data, as well as the relevance of data from different clients. In this article, we investigate the client and data selection problem in FL system from both system and individual perspectives, while considering the impacts of data quality and data relevance. Specifically, from the system perspective, we establish a centralized optimization problem that aims to optimize social welfare in a centralized manner. That is, a central controller decides on the amount of each client's data to be utilized for the FL system, aiming at maximizing the overall social welfare. From the individual perspective, we formulate a two-stage Stackelberg game for incentivizing clients to contribute their data and resources to the FL system in a decentralized manner. In the first stage, the FL server acts as the game leader and specifies a reward mechanism. In the second stage, each client acts as a game follower and competes for the reward by deciding on the amount of data to contribute to the FL system, aiming at maximizing its individual payoff. We analyze the centralized optimization problem and the Stackelberg game systematically for both low and high data relevance scenarios. In particular, we derive the closed-form optimal solution and game equilibrium for the low data relevance scenario, and propose iterative algorithms that effectively converge to a suboptimal solution and subgame equilibrium for the high data relevance scenario. Simulation results verify that data relevance has a significant negative impact on the system performance. That is, the social welfare achieved through centralized optimization and distributed Stackelberg game approaches in the high data relevance scenario is only 19.7% and 24.0%, respectively, of those achieved in the low data relevance scenario.