Using Arbitrary Memory Regions for SCI Communication
Friedrich Seifert, Joachim Worringen, Wolfgang Rehm · 2001
The gap between the memory bandwidth and the bandwidth of modern high--performance interconnects is getting smaller. This increases the relative performance penalty for each copy operation on a data buffer to be transferred between two user processes on different nodes of a cluster. The optimal solution is direct data transfer between the source and destination buffers, without any in--between copy operations, so--called zero--copy transfer. Support for this zero--copy transfer mode is inherent in the design of the SCI interconnect as its communication is based on transparent remote read and write accesses. However, with the available SCI driver software, the user needs to allocate a designated SCI shared memory segment which then can be exported to or which represents remote memory imported from other nodes. In this paper, we present a solution which allows to use arbitrary regions of a process' address space for SCI communication by mapping it into the global SCI address space. Such a technique allows advanced and efficient usage models of SCI like true zero--copying in communication libraries. Keywords---SCI, interconnect, Linux, memory--locking, zero--copy