Communication Efficient Distributed Learning Using Variational Auto Encoders
Anirudh Kasturi, S Vishwamithra, Chittaranjan Hota · 2022
Advancements in mobile technology led to the manufacturing of smart devices that have started generating massive amounts of data. This triggered a new interest in developing intelligent applications that can harness this data's power and provide meaningful insights from it. This data from various devices needs to be aggregated at a central location to provide more accurate predictions, raising severe bandwidth, privacy, and latency issues. This paper presents a novel distributed learning approach that trains a Variational Auto Encoder at each client and then uses this model's generative capabilities to create a sample set of points on the server. The server aggregates these samples from all the clients and then trains a single global model whose parameters are communicated back to the clients. We show through our results that the amount of data transmitted between clients and the server is significantly lower when compared to the federated learning as well as a central learning approach. We also show that our approach outperforms the federated learning approach in terms of accuracy by more than 3% when the data is distributed in a non-iid fashion and at the same time sees a small drop in accuracy which less than 2% in an iid scenario when compared to federated learning and central learning methods. We have also shown how the amount data transmitted and the variation in accuracies when the number of clients vary between 2, 5 and 10.