Communication-efficient and Decentralized Federated Minimax Optimization
Sajjad Ghiasvand, Amirhossein Reisizadeh, Mahnoosh Alizadeh, Ramtin Pedarsani · 2024
As distributed learning applications like Federated Learning, the Internet of Things (IoT), and Edge Computing expand, addressing their limitations becomes crucial. We approach decentralized learning across a network of communicating clients or nodes, focusing on two primary challenges: data heterogeneity and adversarial robustness. To address these, we introduce a decentralized minimax optimization method incorporating two key components: local updates and gradient tracking. Minimax optimization serves as a fundamental tool for adversarial training, ensuring robustness. Local updates are vital in Federated Learning (FL) to alleviate the communication bottleneck, while gradient tracking is necessary to demonstrate convergence amid data heterogeneity. Our analysis of the proposed algorithm, Dec-Fed Track, in nonconvex-strongly-concave minimax optimization demonstrates its convergence to a stationary point. Additionally, numerical experiments support our theoretical results.