Shared Memory Programming on NUMA-based Clusters using a General and Open Hybrid Hardware/Software Approach
Martin Schulz · mediaTUM – the media and publications repository of the Technical University Munich (Technical University Munich) · 2001
The widespread use of shared memory programming for High Performance Computing (HPC) is currently hindered by two main factors: the limited scalability of architectures with hardware support for shared memory and the abundance of existing programming models.In order to solve these issues, a comprehensive shared memory framework needs to be created which enables the use of shared memory on top of more scalable architectures and which provides a user-friendly solution to deal with the various different programming models.Driven by the first issue, a large number of so-called SoftWare Distributed Shared Memory (SW-DSM) systems have been developed.These systems rely solely on software components to create a transparent global virtual memory abstraction on highly scalable, loosely coupled architectures without any direct hardware support for shared memory.However, they are often affected by inherent performance problems and, in addition, do not solve the second issue of the existence of (too) many shared memory programming models.On the contrary, the large amount of work done in the DSM area has led to a significant number of independent systems, each with its own API, thereby further worsening the situation.The work presented within this thesis therefore takes the idea of SW-DSM systems a step further by proposing a general and open shared memory framework called HAMSTER (Hybrid-dsm based Adaptive and Modular Shared memory archiTEctuRe).Instead of being fixed to a single shared memory programming model or API, this framework provides a comprehensive set of shared memory services enabling the implementation of almost any shared memory programming model on top of a single core.These services are designed in a way that minimizes the complexity for target programming models making the implementation of a large number of different models feasible.This can include both existing and new application or application domain specific programming models easing both the porting of given and the parallelization of new applications.In addition, the HAMSTER framework avoids typical performance problems of SW-DSM systems by relying on so-called NUMA (Non-Uniform Memory Access) architectures which combine scalability and cost effectiveness with limited support for shared memory in the form of non-cache coherent hardware DSM.Their capabilities are directly exploited by a new type of hybrid hardware/software DSM system, the core of the HAM-STER framework.This Hybrid-DSM approach closes the semantic gap between the global physical memory provided by the underlying hardware and the global virtual memory rei and NEPHEW (EP 29907).In addition, the many partners Europe-wide involved in these projects provided an additional source of inspiration and support.Special thanks go to Dolphin ICS, most specifically to Kåre Løchson, Hugo Kohmann, Torsten Amundsen, and Roy Nordstrøm, who were involved not only in these ESPRIT projects, but also technically supported our work within SMiLE.The work within NEPHEW also brought me in contact with the Clinic for Nuclear Medicine at the "Klinikum Rechts der Isar".From this cooperation I received valuable input for the evaluation part of this work.This would not have been possible without help from Frank Munz, Sibylle Ziegler, and Martin Völk and I am very thankful for the time and energy they invested.On a more personal note, I am indebted to my friends and my whole family who were always at my side providing constant encouragement and support.I would especially like to thank my good friend, Dr. Johannes Zimmer