Flux: Fine-Grained Communication Scheduling for Distributed Training in Multi-Tenant AI Clusters
Jiashuo Lin, Xingbo Feng, Huilin Qi, Yan Liu, Chenxi Ling, Bo Tang, Yi Wang, Xiaofeng Tao, Weichao Li · 2025
Communication overhead is a major bottleneck in distributed AI training, particularly in multi-tenant environments, limiting GPU utilization. Existing job-level scheduling methods fail to address the varying urgency of individual communication operations. We propose Flux, a novel fine-grained scheduler that prioritizes communication operations based on their Urgency Score and job intensity. Our evaluation shows Flux improves GPU utilization by up to 10 % compared to state-of-the-art job-level algorithms. This demonstrates the significant advantage of fine-grained communication scheduling in multitenant AI clusters.