Mahout on heterogeneous clusters using HadoopCL
Xiangyu Li, Max Grossman, David R Kaeli · 2015
MapReduce is a programming model capable of processing large amounts of data in parallel across hundreds of compute nodes in a cluster. Many applications have leveraged the power of this model, including a number of compute-hungry applications in machine learning. MapReduce can meet the demands of massive data analysis in applications such as web search, digital media selection, and online shopping analytics. The Mahout recommendation system is one of the most popular open source recommendation systems that employs machine learning techniques. Mahout provides us with a parallel computing infrastructure that can be applied to implement a range of applications that work with large datasets. A complementary trend in cluster computing has been the use of GPUs to increase the performance of data-intensive applications. There have been several efforts to utilize GPUs to accelerate the MapReduce framework to improve performance. HadoopCL is a framework that auto-generates OpenCL kernels from Hadoop tasks and can then run them across a heterogeneous cluster. In this paper, we analyze the performance of a Mahout recommendation system running on different cluster platforms, including CPU-only platforms and heterogeneous platforms, where both discrete GPUs and integrated APUs can be evaluated. We propose a cooperative HadoopCL model that improves both GPU/APU programming flexibility and performance.