Seamless Fail-over for PCIe Switched Networks
William Tu, Tzi‐cker Chiueh · 2018
PCI Express (PCIe) was originally designed as a local bus interconnect technology for connecting CPUs, GPUs and I/O devices inside a machine, and has since been enhanced to be a full-blown switched network that features point-to-point links, hop-by-hop flow control, end-to-end retransmission, etc. Recently, researchers have further extended PCIe into an intra-rack interconnect designed to connect multiple hosts within the same rack. To viably apply PCIe to such use cases, additional fail-over mechanisms are needed to ensure continued network operation in the presence of control plane and data plane failures. This paper presents the design, implementation and preliminary evaluation of a fault-tolerant PCIe-based rack area network architecture called Ladon, which incorporates a fail-over mechanism that takes effective advantage of PCIe architectural features to significantly reduce the service disruption time due to a control plane failure of a PCIe switch. Empirical tests on an operational Ladon prototype show that the proposed mechanism ensures that a PCIe root complex failure has zero impact on the data plane and incurs only a modest disruption time (less than 40 sec) for the control plane services.