Distributed Collaborative Filtering on a Single Chip Cloud Computer
Aalap Tripathy, Atish Patra, Suneil Mohan, Rabi Mahapatra · 2013
Many-cores on chip have now become a reality. They necessitate the revisit of several layers of a cloud infrastructure. For this to happen, parallel programming runtimes need to be designed for many-cores on chip as the target architecture. In this paper, we show that Map Reduce programming paradigm can be adapted to run on Intel's experimental single chip cloud computer (SCC) with 48-cores on chip. We demonstrate this using a Collaborative Filtering (CF) recommender system as an application. CF is widely used in e-commerce deployments to predict user's preference towards an unknown item from their past ratings. We address scalability with data partitioning, combining and sorting algorithms, maximize data locality to minimize communication cost within the SCC cores. We demonstrate ~2x speedup, ~94% lower power consumption for benchmark workloads as compared to a distributed cluster multi-processor nodes in use today.