Fine-Grained Visual Aspects in Genre Prediction
Prabir Mondal, Kushum, Tanvi Kumari, Sriparna Saha, Jyoti Prakash Singh, Provat Biswas, Brijraj Singh, Niranjan Pedanekar · IEEE Transactions on Computational Social Systems · 2025
In this study, we investigate the role of visual content in accurately predicting movie genres. By extracting keyframes from Hindi, Bengali, Malayalam, and Telugu language-based Indian movie trailers in the Flickscore dataset, we analyse visual elements using visual language model (VLM) and Large Language Models (LLMs). Our approach focuses on the FAMOS aspects (focus, action, mood, object, setting) to understand the movie’s theme, summary, and genre. This method eliminates the need to manually prepare textual metadata, thus reducing the time and effort required for genre identification. The visual features captured from the keyframes are leveraged to predict genres more efficiently, offering a reliable way to automate this process. Additionally, we integrate the visual information into a content-based filtering (CBF) system to predict user preferences. Our study highlights the effectiveness of using visual features, which significantly enhances the performance of recommendation systems (RS) by improving accuracy in predicting user preferences based on movie genres. We demonstrate that by analysing the implicit content in movie frames, we can achieve better insights than traditional metadata-driven approaches. Overall, our findings emphasise that visual content holds valuable information for understanding movie genres and can play a critical role in user preference understanding.