An Aesthetic-Driven Approach to Unsupervised Video Summarization
Hongben Huang, Zaiqun Wu, Guangyao Pang, Jiehang Xie · IEEE Access · 2024
The aim of video summarization is to condense lengthy videos into shorter versions, making them more accessible for viewing. Typically, people can identify important shots within a video by using audiovisual cues and assessing the aesthetic attributes of the frames. However, existing methods either focus only on unimodal features or neglect the aesthetic attributes of videos, resulting in the limited quality of the generated summaries. Particularly, the reliance on annotated data for training models also imposes limitations, as it not only demands significant time and resources but may not capture the diverse and subjective nature across different videos. To tackle these issues, we propose an aesthetic-driven approach to unsupervised video summarization, namely ADUVS. Specifically, ADUVS incorporates an aesthetics encoder to extract key aesthetic attributes. Additionally, we design a multimodal fusion module that assesses how different modalities of information complement each other and highlights the most relevant segments for the desired summary. Moreover, the training process for ADUVS does not require reliance on annotated data, thus reducing both time and labor costs. Extensive experiments demonstrate that our proposed method is better than various benchmark methods across commonly used evaluation metrics.