Towards Multimodal Search and Visualization of Movies Based on Emotions
Francisco Caldeira, João Lourenço, Nuno Tavares Silva, Teresa Chambel · ACM International Conference on Interactive Media Experiences · 2022
Movies are one of the most important and impactful forms of entertainment and a powerful vehicle for culture and education, due to the cognitive and emotional impact on the viewers, and technology has been making them more accessible in pervasive services and devices. As such, the huge amount of movies we can access, and the important role emotions play in our lives, make more pertinent the ability to access, visualize and search movies based on their emotional impact. In this paper, we characterize the challenges and approaches in this scenario, and present interactive means to visualize and search movies based on their dominant and actual emotional impact along the movie, with different models and modalities. In particular through emotional highlights and trajectories, the user's emotional state, or a music being played. Music contributes greatly to the emotional impact of movies and it can also be a trigger to get us into one of them in serendipitous moments.