Static Yet Flexible: Expander Data Center Network Fabrics
Simon Kassing · Repository for Publications and Research Data (ETH Zurich) · 2017
Recent work has indicated that any static data center network is fundamentally limited, due to its inability to move around network capacity.Is this truly the case, is the static network not flexible enough to handle varying (skewed) traffic scenarios through only traffic engineering?Can we only find refuge in dynamic topologies, introducing on-the-fly re-arrangement of network links at a cost?In pursuit of these research goals, three main data center topologies were evaluated: traditional (oversubscribed) fat-trees, expanders and dynamic topologies.Using flow optimality evaluation via a linear program, worst case traffic scenarios were identified and their respective performance measured under perfect traffic engineering.A custom discrete packet simulator was used to evaluate the two static topologies under a wide range of traffic scenarios and compare results to performance claims of recent dynamic topology research.It was found that the modeling of server up-links is crucial to a meaningful comparison.Equivalent performance to that of the most recent dynamic topology was achieved.At 67.5-80% of its cost, the expander was able to rival the fat-tree in all traffic scenarios using at least one of its routing strategies.Three factors had significant impact in this: (a) the high ToR-ratio (per-ToR network up-links divided by per-ToR server up-links), (b) the low average shortest path length, and (c) the property that every cut in the network is traversed by many links.It was found that oversubscription of the fat-tree indeed creates problematic traffic scenarios, where communication between only a small fraction of active servers is severely impaired.The work raises new potential inquiries pertaining to the vast space of routing logic and their parametrization, engineering challenges in the deployment of expanders, exploration and topology-independent characterization of workload in the data center, further verification through use of other simulators or physical deployment, and theoretical understanding of the capabilities of dynamic topologies.