Machine learning-based product recommendation using Apache Spark
Lin Chen, Rui Li, Yige Liu, Ruixuan Zhang, Diane Myung-kyung Woodbridge · 2017
There have been growing interests in the area of recommender systems using machine learning techniques. As there are a great number of explicit and implicit features that can be used for estimating user preference, it requires scalable and accurate algorithms along with a system with high availability and scalability. Alternating least square matrix (ALS) algorithm is an enhanced version of latent factor models using matrix factorization with good scalability and predictive accuracy. Apache Spark is an open-source distributed platform for processing big data, achieving good speed and scalability suitable for iterative machine learning algorithms. Amazon offers cloud computing services with various functionality including data storage and processing engines and is highly available and scalable. In this study, we applied the ALS algorithm using Apache Spark running on an Amazon Web Service (AWS) Elastic Map Reduce (EMR) cluster for recommending a product with a good accuracy and enhanced scalability.