A Random Access based Approach to Communication-Efficient Distributed SGD
Jinho Choi · 2022
In this paper, we study communication-efficient distributed stochastic gradient descent (SGD) with data sets of users distributed over a certain area and communicating through wireless channels. Since the time for one iteration in the proposed approach is independent of the number of users, it is well-suited to distributed SGD with a large number of users or devices. Furthermore, since the proposed approach is based on random access in machine-type communication (MTC), it can be easily employed for training models with a large number of devices in various Internet-of-Things (IoT) applications where MTC is used for their connectivity. For fading channel, we show that noncoherent combining can be used. As a result, no channel state information (CSI) estimation is required.