Revisiting Overlay Multicasting for the Cloud
Karthik Nagaraj, Hitesh Khandelwal, Charles Edwin Killian, Ramana Rao Kompella · 2010
Group communication systems (e.g., multicast, DHTs) have emerged as basic primitives for several large-scale distributed systems. Most existing systems that implement these primitives often assume a flat topology of overlay nodes. For instance, many DHTs assume that all overlay links are often homogeneous in their capacities, costs and other such characteristics. Similarly, multicast protocols create overlay trees without taking into account the physical location of nodes. While a few systems do consider latency between different overlay nodes, the fact that metrics such as latency change continuously often translates into additional complexity in constantly measuring and reorganizing nodes. Modern trends in hierarchical data center designs and global services running across several geographically disparate data centers pose unique challenges and opportunity to revisit the design of group communication systems with location awareness in-built into these systems from the ground up. In this paper, we present the design and architecture of one such system called DC2, in which nodes are aware of their location within the data center (e.g., rack, aggregation switch) and organize group communications in order to minimize the expensive cross-data center links or cross-hierarchy links as much as possible. In our experiments using a real prototype deployed over 700 virtual nodes running over 15 physical machines, we found that DC2 minimizes message latencies by several orders of magnitude, and reduces node and link stress by a factor of 2 to 3 ×.