Programming models for computers of extreme scale parallelism
Yugendra R. Guvvala · ThinkTech (Texas Tech University) · 2013
Computational speeds of processors (CPUs) have been increasing following Moors law and recently due to multicore machines. But, we are not harnessing the adequate power of processors if we are not taking into account other components such as communication bandwidths, storage media, interconnects, programming and memory architectures, which could also have significant impact in determining speed of computational resource. Some of the major barriers of high computing are dominated by latencies incurred due to storage, system tasks, communication, and component failures. High Performance Computing is trending towards exa-FLOPs and these barriers will be very critical in next generation exascale computing as they might prove to be very expensive. In the next generation exascale-computing era the compute nodes are branching into two different categories: Homogeneous and Heterogeneous architectures. In this study we identify potential problems faced in both architectures and discuss techniques to overcome latencies incurred due to communication and thread creation on many core machines. Programming models discussed here are developed using message passing communication for homogeneous architectures and POSIX thread for heterogeneous architecture. The technique discussed in this study for homogeneous architecture is Communication Delegation Model, which will provide an efficient way of communicating across nodes and reducing communication channel resource contention. This method proves to be very powerful as it shifts communication overhead on all the cores to dedicated communication cores. The programming technique discussed for heterogeneous architectures is Thread Optimization Model. This model reduces non-data overhead incurred due to thread creation and termination on multicore architectures. Experimental results have proven that these studies would provide a huge benefit for exascale machines, which we will be looking at in near future.