Market Basket Analysis Algorithm with Map/Reduce of Cloud Computing
Jongwook Woo, Yuhang Xu · 2012
Abstract – Map/Reduce approach has been popular in order to compute huge volumes of data since Google implemented its platform on Google Distributed File Systems (GFS) and then Amazon Web Service (AWS) provides its services with Apache Hadoop platform. Map/Reduce motivates to redesign and convert the existing sequential algorithms to Map/Reduce algorithms for big data so that the paper presents Market Basket Analysis algorithm with Map/Reduce, one of popular data mining algorithms. The algorithm is to sort data set and to convert it to (key, value) pair to fit with Map/Reduce. It is executed on Amazon EC2 Map/Reduce platform. The experimental results show that the code with Map/Reduce increases the performance as adding more nodes but at a certain point, there is a bottle-neck that does not allow the performance gain. It is believed that the operations of distributing, aggregating, and reducing data in Map/Reduce should cause the bottle-neck.