All routes to efficient datacenter fabrics
Nikolaos I. Chrysos, Fredy Neeser, Mitch Gusat, Cyriel Minkenberg, Wolfgang E. Denzel, Claude Basso · 2014
Performance optimized datacenters (PoDs) require efficient PoD interconnects to deal with the increasing volumes of inter-server (east-west) traffic. To cope with these stringent traffic patterns, datacenter networks are abandoning the oversubscribed topologies of the past, and move towards full-bisection fat-tree fabrics. However, these fabrics typically employ either single-path or coarse-grained (flow-level) multi-path routing. In this paper, we use computer simulations and analysis to characterize the waste of bandwidth that is due to routing inefficiencies. Our analysis suggests that, under a randomly selected permutation, the expected throughputs of d-mod-k routing and of flow-level multi-path routing are close to 63% and 47%, respectively. Furthermore, nearly 30% of the flows are expected to undergo an unnecessary 3-fold slowdown. By contrast, packet-level multi-path routing consistently delivers full throughput to all flows, and proactively avoids internal hotspots, thus serving better the growing demands of inter-server (east-west) traffic.