Review-Enhanced Universal Sequence Representation Learning for Recommender Systems
Junjie Zhang, Wenqi Sun, Yupeng Hou, Wayne Xin Zhao, Ji-Rong Wen · ACM Transactions on Information Systems · 2025
With the widespread deployment of recommender systems on various online platforms, researchers are striving to develop transferable recommendation algorithms that can effectively adapt to new task scenarios without requiring the re-training of new recommenders. However, there have been challenges in dealing with explicit ID modeling in this context. Recently, researchers have drawn inspiration from the achievements of pre-trained language models (PLMs), making it possible to acquire ID-agnostic representations by utilizing the corresponding texts of items. These representations have shown to be transferable across diverse domains. However, while these methods demonstrate generalization, they are less proficient in making personalized recommendations as they learn universal representation. In light of this issue, we present a review-enhanced universal sequence representation learning approach named RUNSRec . Our goal is to not only comprehend universal user behavioral patterns across different domains but also capture their inherent preferences to make recommendations. Our approach makes three technical advancements toward this objective. Firstly, we introduce a lightweight item encoding architecture based on parametric whitening and mixture-of-experts enhanced adapter. It learns discriminative item textual representations by encoding their corresponding identity text and review text, with a discriminative keyword extraction method to enhance the representation identifiability. Secondly, we propose a universal sequence representation learning method that enables the training of transferable recommenders across diverse domains, based on two novel contrastive learning tasks. Furthermore, we introduce a personalized adapter tuning mechanism that enables the universal recommender to capture user personal preferences in a parameter-efficient way. By incorporating universal behavioral patterns learned during the pre-training stage and personalized user tastes captured through adapter tuning, our approach achieves a better balance between generalization and personalization in transferable recommender systems. Extensive experiments conducted on five real-world datasets have demonstrated the effectiveness of our proposed approach.