Fair Coflow Scheduling without Prior Knowledge
Luping Wang, Wei Wang · 2018
Coflow scheduling improves the networking performance at the application level in datacenters. Ideally, a coflow scheduler should provide tenants with isolation guarantees to achieve predictable networking performance. Existing works in this regard (e.g., DRF [1] and HUG [2]) are limited to the clairvoyant scheduling, in that the complete knowledge of coflow sizes is assumed to be available before the communication starts. However, this assumption does not hold for many applications with pipelined computation, in which clairvoyant coflow schedulers become inapplicable. To bridge this gap, we develop a new non-clairvoyant coflow scheduler, called Non-Clairvoyant DRF (NC-DRF), which provides isolation guarantees between contending coflows without prior knowledge of coflow size. We show that NC-DRF achieves provable isolation guarantees in the long run. Cluster deployment and trace-driven simulations show that with NC-DRF, coflows are only delayed by 68% on average as compared with the clairvoyant, isolation-optimal DRF [1]. NC-DRF also outperforms existing alternatives (e.g., per-link fairness) by 1.7× in terms of the average coflow completion time.