On the Necessity of Aligning Gradients for Wireless Federated Learning
Wei-Ting Chang, Mohamed Seif, Ravi Tandon · 2021
In this paper, we consider the problem of wireless federated learning, where the users wish to jointly train a machine learning model with the help of a parameter server. During the training, the local gradients from the users are aggregated over a wireless channel. Typically, coefficients of the local gradients are aligned by power control techniques to ensure that the estimated aggregated gradient is an unbiased estimator of the true gradient. However, schemes that align gradients require coordination, can be challenging to implement in practice, and often lead to degraded performance due to heterogeneity of users’ channels. In this paper, we show that alignment of gradients for wireless FL is not always necessary for convergence. Specifically, we consider non-convex loss functions, and derive conditions under which misaligned wireless gradient aggregation still converges to a stationary point. We also present experimental results to show that transmitting at full power can outperform aligned gradient aggregation depending on the heterogeneity of users’ channels.