MapReduce Accounting System Integrated with High-Performance Computing Infrastructure
Chia‐Chuan Chuang · 2015
Many service providers have the capacity to immediately build a Hadoop platform on top of a virtual-machine cluster when a provision request arrives. However, the input/output intensive property of MapReduce dramatically harms virtual-machine job performance and lengthens the duration of job execution. In addition, fee of virtual-machine-based platforms is charged according to the uptime of created virtual machines, instead of by job execution duration. This study deployed a physical Hadoop big data platform for public use, and implemented an accounting system to accurately calculate consumed computing resource. Furthermore, this big data platform is integrated with a high-performance computing infrastructure that has operated for decades at National Center for High-performance Computing, Taiwan.