Paradigm Shift for EXASCALE Computing
George Matheou, Paraskevas Evripidou, Costas Kyriacou · 2015
In this paper we propose a paradigm shift for exascale computing by using a Hybrid Data-Flow/Control-Flow model of execution. Programming of High Performance Computers is mainly done through parallel extension of the sequential model like MPI and OpenMP. Even though these extensions facilitate high productivity parallel programming, they suffer from the inability to tolerate long latencies. The Data-Flow model of execution enforces only a partial ordering as dictated by the true data-dependencies. This is very beneficial for parallel processing because it allows to exploit the maximum parallelism. Furthermore it tolerates synchronization and communication latencies. We believe that a paradigm shift to a hybrid Data-Flow and Control-Flow system will improve the performance of High Performance Computing (HPC). Data Driven Multithreading (DDM), a threaded Data-Flow programming/execution model, could be the platform for the HPC paradigm shift. Our work on DDM showed that DDM can efficiently run on state-of-the-art sequential machines, resulting in a Hybrid Data-Flow/Control-Flow system. Evaluation results of DDM implementations on a variety of platforms showed that DDM can indeed tolerate synchronization and communication latency. When comparing DDM with OpenMP, DDM performed better for all benchmarks used. This is primarily due to the fact that DDM effectively tolerates latency. Similar results were also obtained when comparing DDM implemented on a Cell processor, with CellSs and Sequoia.