Enhancing Multimodal Recommender System Performance Using Large Language Models
Soohyun Woo, Seonu Park, Xinzhe Li, Seongkook Lee, Junsu Kim · IEEE Access · 2026
With the rapid advancement of e-commerce platforms and the overwhelming amount of information available online, recommender systems have emerged as essential tools for helping users efficiently discover items aligned with their preferences. Online reviews, in particular, offer rich information for understanding both user preferences and item features, and have been widely utilized in existing multimodal recommender systems. However, most prior approaches have not sufficiently addressed modality-specific noise in multimodal reviews, thereby constraining their capacity to precisely represent users’ experiences. Furthermore, by merely extracting and fusing features from texts and images without considering the structural heterogeneity between modalities, these approaches often fall short in accurately predicting user preferences. To address these limitations, this study proposes multimodal aspect summarization for recommender systems (MAS-Rec), a novel model that leverages large language models (LLMs) to summarize multimodal reviews by aspect, thereby enhancing preference prediction. The proposed model is structured as follows: First, LLMs are employed to independently summarize user preferences and item features into aspects from textual and visual data, respectively. Second, these aspect-based summarizations are then encoded using a bidirectional encoder representations from transformers (BERT) to extract semantic features. Finally, a co-attention mechanism is applied to model the semantic interactions between user preferences and item features. Experimental results on two review datasets provided by Amazon.com demonstrate that MAS-Rec outperforms existing unimodal and multimodal models. These findings demonstrate that summarizing images by aspect using the LLM effectively captures the semantic connections between texts and images, thereby enabling more accurate prediction of user preferences compared to traditional approaches.