MULTIMODAL CONTENT-BASED RECOMMENDER SYSTEM USING THREE-DIMENSION CONVOLUTION NEURAL NETWORK
Khamael Raqim Raheem, Israa Hadi Ali · Zenodo (CERN European Organization for Nuclear Research) · 2020
Research on Recommender Systems has grown tremendously over the past few years; however, the quest to date for how user emotions can be used as implicit feedback to supplement these systems is sparse. Recommender Systems should take advantage of the high availability of digital data to collect input data of various types that allow the system to enhance its accuracy implicitly or explicitly. In this study, a Multimodal Content-Based Recommender System for image recommendation is proposed which is based on Implicit and Explicit Feedbacks. In order to obtain the Implicit Feedbacks, a Convolution Neural Network with Three-Dimensions is constructed to predict the emotion of the user's face if it is positive or negative. The Convolution Neural Network making a mixture of spatial and temporal data in Three-Dimension Convolution in order to learn about a transition in consecutive frames. The results of predictions of Neural Network are used as Implicit Feedback for the recommendation algorithm. The Multimodal Recommender System is built by combining the output of two Content-based Recommender Systems using a binary Logistic Regression algorithm. Content-based Recommender System is built by training the Support Vector Machine classifier on features of item profile and Explicit or Implicit feedback. The performance measures are computed based on predicted and ground truth feedbacks. The result shows that the Three-Dimension Convolution Neural Network contributes to Implicit Feedbacks prediction in the Recommender System. Also, the combination of the results of two Recommender Systems with different feedback techniques can enhance the performance of the proposed system.