From In-Network Allreduce to in-Nic Allreduce in Support of Optically Switched Gpu Networks
Libin Wang, Hongxiang Guo, Cen Wang, Yuepeng Wu, Xiyang Lan, Jian Wu · 2025
We propose in-NIC AllReduce to lower communication times, adapting to optically switched GPU network for distributed training. The experimental results show a$1.89 \times$acceleration aligning with a theoretical analysis of$1.90 \times$acceleration on average.