Fast Synchronization of Model Updates for Collaborative Learning in Micro-Clouds
Long Luo, Yuchen Zhang, Qixuan Jin, Hongfang Yu, Gang Sun, Shouxi Luo · 2021
Micro-clouds are increasingly used to carry out collaborative distributed machine learning, avoiding the cost, performance, and privacy issues of traditional cloud-centric learning due to the centralization of large amounts of data to the cloud through wide-area networks (WANs). Synchronizing model updates plays an important role in such a global collaborative learning system. However, due to the lack of flexibility, existing solutions may result in slow completion of data transmission for model updates synchronization among mirco-clouds. This paper proposes an efficient solution (Amusync) to achieve fast model updates synchronization via flexible hierarchical aggregation for model updates and adaptive routing for model data transfers. Amusync optimizes the selection for aggregators and routing paths according to network condition and traffic loads of model updates synchronization and uses an optimization-based algorithm to quickly find solutions. We conduct extensive simulations on real-world network topologies, and the results show that Amusync can speed up the parameter aggregation and distribution by reducing by as much as around 90%.