Design and implementation of high performance communication subsystems for clusters
Dhabaleswar K. DK Panda, Mohammad Banikazemi · 2000
With the significant increase in computing power of processors and tremendous improvement in the performance of networking hardware, clusters have become a popular platform for high performance computing. In order to make the performance of clusters comparable to that of traditional high performance computing systems, it is crucial to make the communication subsystems of these systems as efficient as possible. In recent years, communication subsystems with user-level protocols have been proposed by the research community and industry to address this issue. All of these communication systems use much simpler communication protocols in comparison with legacy protocols such as the TCP/IP. The role of the operating system has been much reduced in these systems and in most cases user applications are given direct access to the network interface. The Virtual Interface Architecture (VIA) specification has been developed to standardize these user-level protocols and to make their ideas available in commercial systems. The primary objective of this research is to design and implement efficient and high performance communication subsystems for clusters with user-level protocols such that the high performance of the networking technologies is passed to applications. To achieve this goal, this thesis is focused on five components of communication subsystems: network interface support, communication mechanism, distribute shared memory (DSM) support, distributed memory support, and performance evaluation. Several design choices for various components of VIA are proposed and evaluated on different platforms. A prototype implementation of VIA is developed for IBM SP-connected clusters. This implementation remains to be the most efficient software implementation of VIA to date. The performance of this implementation is extensively evaluated and performance bottlenecks have been identified. Furthermore, several hardware enhancements for improving the performance are studied. Design and implementation of the communication infrastructure required for supporting distributed shared memory and distributed memory programming models on top of user-level communication protocols are also studied. The proposed communication mechanisms and their extensive evaluation demonstrate significant potential to be applied to the design of communication subsystems for current and future clusters.