High Performance Computing and Big Data Convergence: A Technical Review
Mirtaheri Seyedeh Leili, Grandinetti Lucio · Advances in parallel computing · 2018
The convergence of High Performance Computing (HPC) and Big Data Analytics (BDA) has been the center of attention for past few years. HPC and BDA have separate software stacks and from financial point, it is impossible to invest in both categories at the same time. HPC is traditionally compute intensive while BDA is data intensive. In this paper we investigate the convergence of HPC and BDA from a technical perspective. We first review the state of the art and introduce the common frameworks of HPC and BDA. We also compare these frameworks in terms of scalability, data rate, data size, fault tolerance and real-time processing. We further compare the software stacks for HPC and BDA as convergence challenges and discuss on existing solutions to convergence.