Buffer architecture for Dragonfly topologies
Cano, Alejandro, Camarero, Cristóbal, Martínez, Carmen, Beivide, Ramón · UPCommons institutional repository (Universitat Politècnica de Catalunya) · 2023
An interconnection network comprises routers, links, and servers, and the arrangement of these elements is known as network topology. Various network topologies have been employed for supercomputers, including Dragonfly [1], Dragonfly+, Fat-tree, etc. Each topology presents advantages and disadvantages in terms of performance, cost, and scalability, with different routing mechanisms employed for each one. The Dragonfly topology is composed of a global complete graph connecting super-nodes or groups of switches, which, in turn, are connected by means of local complete graphs. Thus, it is a two-level hierarchical network with diameter three, or the distance between any pair of switches is three. A small instance of a Dragonfly topology is showed in Figure 1. The Frontier supercomputer, currently number one in the TOP500 list, employs a Dragonfly of this type. Apart from the topology, another crucial part in the design of interconnection networks is the routing mechanism. Usually, adaptive routing is employed so a packet can choose between following a minimal route, minimizing the number of hops of a packet over the network, or a non-minimal route, which usually follows the Valiant routing scheme. Specifically, in Dragonfly, minimal paths are of type local-global-local (hereinafter referred to as lgl). These paths start with a local link inside the source group, then a global link to achieve the destination group, and finally a local link to arrive to the destination switch; with any of the links being possibly omitted depending on the relative locations of the source and destination switches. For non-minimal routes, initially, a Valiant routing variant was proposed. This variant selects an intermediate group of the Dragonfly to pass through before approaching the destination switch, making lg-lgl routes. However, it was discovered that Valiant routes should select a switch of the network instead of a group, due to over-subscription of the local links in some patterns, and therefore lgl-lgl routes should be used, composing Valiant routes of two equal phases [2]. Also, the routing mechanism should be designed to not to introduce cyclic buffer dependencies in the network, which could lead to packet-deadlock. Therefore, a buffer architecture should be employed to avoid this phenomenon, which is responsible for assigning a buffer (or Virtual Channel, VC) to each packet at each hop across the network.