Designing High Performance and Scalable Unified Communication Runtime (UCR) for HPC and Big Data Middleware
Jithin Jose · OhioLink ETD Center (Ohio Library and Information Network) · 2014
The computation and communication requirements of modern HighPerformance Computing (HPC) and Big Data applications are steadily increasing.HPC scientific applications typically use Message Passing Interface (MPI) as the programming model, however, there is an increased focus on hybrid MPI+PGAS (Partitioned Global Address Space) models for emerging exascale systems.Big Data applications rely on middleware such as Hadoop (including MapReduce, HDFS, HBase, etc.) and Memcached.It is critical that these middleware be designed with high scalability and performance for next generation systems.In order to ensure that HPC and Big Data applications can continue to scale and leverage the capabilities and performance of emerging technologies, a high performance communication runtime is much needed.This thesis focuses on designing a high performance and scalable Unified Communication Runtime (UCR) for HPC and Big Data middleware.In HPC domain, MPI has been the prevailing communication middleware for more than two decades.Even though it has been successful in developing regular and iterative applications, it can be very difficult to use MPI and maintain performance for irregular, data-driven applications.PGAS programming model presents an attractive alternative for designing such applications and provides higher productivity.It is widely believed that parts of applications can be redesigned using PGAS models -leading to hybrid MPI+PGAS applications, and improve performance.In order to fully leverage the performance benefits offered by the modern HPC systems, a unified communication runtime that offers the advantages of Publications