Multi-modal similarity fusion for user behavior sequence modeling
Binglin Li · 2024
With the pervasive use of the Internet, recommendation systems have gained increasing importance in people's daily lives. Among the crucial tasks in recommendation systems, click-through rate prediction stands out as it directly influences their effectiveness. Recent studies have revealed that incorporating user behavior sequences can substantially enhance the accuracy of click-through rate prediction models. However, existing models overlook user preferences for textual and visual information, which hampers the acquisition of a comprehensive representation of user interests. Consequently, this limitation results in suboptimal model accuracy. In this paper, we propose a unified framework for modeling multi-modal user behavior sequences. Our framework leverages a unified cross-modal pre-trained model for feature extraction and employs a multi-modal similarity-enhanced attention mechanism to capture users' preferences across various modalities. We conduct extensive experiments on large-scale real-world datasets to validate the effectiveness of our approach. Compared to other state-of-the-art click-through rate estimation algorithms, our model achieves an approximate 1.22% improvement in AUC, thereby significantly enhancing the accuracy of the recommendation model.