BOAD: Optimizing Distributed Communication with In-Kernel Broadcast and Aggregation
Jianchang Su, Yifan Zhang, Linpu Huang, Wei Zhang · 2024
Efficient communication is crucial for the performance of big data and distributed computing systems. Two key communication patterns in these systems are broadcasting, which involves sending data from one to multiple nodes, and aggregation, which combines data from multiple nodes into a single result. Traditional methods using socket-based communication often suffer from significant latency due to frequent user-kernel crossing and network stack processing, limiting the scalability and efficiency of these systems.