Tree Network Design for Faster Distributed Machine Learning Process with Distributed Dual Coordinate Ascent
Myung Cho, Meghana Chikkam, Weiyu Xu, Lifeng Lai · 2024
This paper delves into the subject of designing a tree network, enabling the application of Distributed Dual Coordinate Ascent on a general tree network (DDCA-Tree) introduced in [1] – [3] for distributed Machine Learning (ML) process. We assume that a network is characterized by communication delays proportional to the distance between any two nodes. To efficiently managing distributed data across the network, we propose the Minimum Worst-Distance Tree (MWDT) algorithm for designing a tree network with a specified target depth yielding a network structure where the communication delay in worst path between a leaf node and its parent node is minimized, consequently enhancing the convergence speed of DDCA-Tree. In numerical experiments, to validate the effectiveness of our approach, we compared the communication delay in worst path on a tree network generated by our algorithm against a minimum spanning tree which provides minimum weight (i.e., distance) sum, and showed our network design has reduced distance in worst path.