Distributed Link Heterogeneity Exploitation for Attention-Weighted Robust Federated Learning in 6G Networks
Qiaomei Han, Xianbin Wang, Weiming Shen · 2024
The rapid evolution of wireless communications is paving the way for distributed computing, enabling pervasive intelligence in 6G networks through distributed machine learning particularly federated learning (FL). Despite the dramatically enhanced connectivity expected from 6G, imperfect or hetero-geneous communication links among distributed participating devices remain as a fundamental challenge for the performance improvement of FL. In overcoming the communication con-straint, existing solutions primarily focus on reducing communication overhead through techniques like FL model compression or sparsification. However, these approaches often ignore the impact of link heterogeneity among distributed devices on FL performance. To bridge this gap, we propose a heterogeneous link attention-weighted FL framework in 6G networks through the characterization of link heterogeneity and the design of an attention mechanism-driven model aggregation method. Specifically, leveraging prior knowledge about distributed communication links and their performance metrics such as latency, reliability, and data rate, the FL server calculates the joint performance dissimilarities among these links, thereby characterizing their heterogeneity relations as a binary probabilistic matrix. Sub-sequently, an attention mechanism is employed for FL model aggregation, where the generated attention weights represent the degree to which each link is influenced by other links. Therefore, the obtained global FL performance can be ensured. Experimental results further demonstrate that our proposed FL framework and model aggregation approach robustly handle FL under the impact of link heterogeneity and optimize the learning performance of FL.