Evaluation of Communication Overhead for Distributed Deep Learning for Local Data Privacy
Yuma Okuda, Akihito Nishikawa, Hiroaki Nishi · 2023
In recent years, there has been growing interest in using big data for machine learning applications. However, as data are often distributed across multiple clients, privacy and physical limitations can make it difficult to aggregate and use these data on centralized servers. Federated learning and split neural networks enable machine learning models to be built without requiring the aggregation of distributed data. These methods have received increased attention in the industrial sector as a means of eliminating impediments to privacy preservation in data sharing across various institutions, such as smart factories. One critical aspect of implementing such methods in practice is analyzing the time variation of traffic and estimating the throughput required for operationalizing machine learning. Previous studies have formulated traffic characteristics without considering packet overhead and measured only the total traffic and the throughput per iteration; such an approach is inadequate from a traffic monitoring perspective. To address these issues, this study conducted an empirical investigation of traffic patterns under realistic conditions, with a particular focus on quantifying traffic and latency. The results of this study suggest that due to the dynamic nature of throughput, it is essential to measure traffic using time units that can accommodate the temporal variability of traffic.