Big data, high-performance computing, and MapReduce
Atanas Radenski · 2014
We discuss the emergence of data-intensive computing and then explore the applicability of business-oriented big-data platforms, such as Hadoop MapReduce, to traditional scientific computing processes. In particular, we investigate the suitability of MapReduce parallelism for simulation of grid-based models by developing message-passing MapReduce algorithms and empirically evaluating their performance on the Amazon's Elastic MapReduce cloud. We outline MapReduce challenges (such as insufficient speed) and opportunities (such as fault-tolerance and ease of use) in scientific computing.