A Dual-Tower Model for Enhanced Sequence Recommendations Using Time-Mixing Attention and Adaptive Gating Mechanism
Zhaowang Wu, Mengfan Yu, Xiaoyang Li, Wei Feng He, Kaixin Deng, Mingyang Tang · 2024
Recent strides in Natural Language Processing (NLP), exemplified by advanced models such as BERT, have significantly boosted capabilities in text analysis and the field of sequence-based recommendation systems. Despite the effectiveness of current models like BERT4Rec, they generally fail to fully capture the nuanced emotional and contextual insights found in user feedback, primarily focusing on past user-item interactions. This study presents the DTM model, a groundbreaking dual-tower framework aimed at overcoming these shortcomings. By merging user-generated content with their interaction histories, DTM leverages a time-mixed attention strategy and dynamic head integration to refine the self-attention process in sequence analysis. Moreover, user feedback is analyzed using a GRU (Gated Recurrent Unit) model equipped with a novel attention mechanism, which enables a responsive representation of emotional and contextual elements in user-item engagements. A sophisticated gating mechanism is then employed to synthesize insights from both sources. Comprehensive testing on various public datasets has shown that DTM surpasses standard models such as BERT4Rec in numerous evaluation criteria, underscoring its superior ability to provide more precise and holistic recommendations.