Top-k User-Based Collaborative Recommendation System Using MapReduce
Sheheeda Manakkadu, Srijan Prasad Joshi, Tom Halverson, Sourav Dutta · 2021 IEEE International Conference on Big Data (Big Data) · 2021
Collaborative Filtering (CF) methods are widely used in recommender systems. However, due to the high computational complexity of CF, the existing algorithms do not scale well when the number of users and items grow. In this paper, we develop a new user-based collaborative filtering method using the MapReduce framework to recommend top-k items for each user in a given dataset. Our proposed method requires a dataset that contains item numbers, user ids, and ratings. We applied our proposed method to recommend movies using the MovieLens dataset. We implement our proposed algorithms on Apache Spark running on Amazon Web Services (AWS) clusters. We perform experiments on different cluster sizes to demonstrate the scalability of our proposed method. The experimental results show that the proposed method is highly scalable and exhibits near-linear speed-up when we increase the number of clusters.