Research on Recommendation Algorithm Based on E-commerce User Behavior Sequence
Li Bai, Allam Maalla, Mingbiao Liang · 2021 IEEE 2nd International Conference on Information Technology, Big Data and Artificial Intelligence (ICIBA) · 2021
With the rapid rise and development of the Internet, e-commerce has become one of the most promising and fastest-growing industries. we propose an architecture of a personalized recommendation system based on offline mining, real-time mining, and deep learning technology. First, through the Flume + Kafka + Spark Streaming system, user behavior data is collected and stored in the business database after data preprocessing First, through the Flume + Kafka + Spark Streaming system, user behavior data is collected and stored in the business database after data preprocessing, which is the work preparing for the next step of data mining. Data mining includes offline mining and real-time mining. Offline mining mainly runs MapReduce on the Hadoop platform, there are also some calculations on the Spark platform, and the calculation results are stored in the business database; The implementation of real-time mining mainly is based on the message subscription of the Kafka cluster, and get real-time consumption statistics through Spark clusters; Deeplearning4j, a deep learning framework, runs on a multi-GPU Spark distributed cluster, which can be optimized the algorithm model online and generate recommendation results to push to the real-time business database and give feedback to the users.; It can also extract features from these data for training automatically, which is conducive to improving the quality and efficiency of users' shopping decisions, and improve the platform's cross-selling capabilities, shorten the user's shopping path, increase traffic conversion rates.