A Case for Flash Memory SSD in Hadoop Applications
Seokhoon Kang, Dong-Hyun Koo, Woon-Hak Kang, Sang-Won Lee · International Journal of Control and Automation · 2013
As the access speed gap between DRAM and storage devices such as hard disk drives is ever widening, the I/O module dominantly becomes the system bottleneck. Meanwhile, the map-reduce parallel programming model has been actively studied for the last few years. In this paper, we will show empirically show that flash memory based SSD(Solid State Drive) is very beneficial when used as local storage devices in IO-intensive map-reduce applications (e.g. sorting) using Hadoop open source platform. Specifically, we present that external sorting algorithm in Hadoop with SSD can outperform the algorithm run with hard disk by more than 3. In addition, we also demonstrate that the power consumption can be drastically reduced when SSDs are used.