Dual Disentangled Attention for Multi-Information Utilization in Sequential Recommendation
Ziqiang Cui, Yixin Su, Fangquan Lin, Cheng Zhong Yang, Hanwei Zhang, Jihai Zhang · 2022 International Joint Conference on Neural Networks (IJCNN) · 2022
Sequential recommendation systems predict the next item that a user will interact with according to the sequential patterns inferred from historical interacted items. Benefiting from the self-attention mechanism, BERT has achieved success in sequential recommendation. Nevertheless, it is still challenging to leverage multiple types of information (e.g., item IDs, attributes, timestamps) effectively under the BERT framework. Existing methods fuse different types of information as the input of BERT by directly adding or concatenating their embedding vectors. This way of fusion ignores the differences of different types of information in sequential pattern inference, which can cause the mutual interference problem and hinder the effective utilization of multi-information. In this work, we propose a Dual Disentangled Attention (DDA) based BERT model, called DDA-BERT, for better leveraging multi-information. Our model performs disentangled information modeling from two aspects: 1) different views to describe items using content information (e.g., using IDs or using attributes) are modeled in different attention heads, respectively; 2) different sequential information (time intervals, relative position distances) modeling is disentangled from content information modeling while calculating the attention scores. As a result, DDA-BERT can effectively prevent the mutual interference problem to improve the prediction accuracy. Extensive experiments on three benchmark datasets show that DDA-BERT consistently outperforms the state-of-the-art baselines by up to 30%.