Video Summarization through Latent Representation: A VAE and KMeans Clustering Method
J.V. Vidhya, R. Annie Uthra · 2024
Video summarization plays a crucial role in various domains such as surveillance, news, search engines, and social media. It entails condensing lengthy videos into concise summaries by selecting a subset of frames that encapsulate the most pertinent information. This study introduces a novel approach to extract key frames from video data employing a combination of a Variational Auto encoder (VAE) and K-Means clustering. Initially, a VAE model is trained on the video frames, with the encoder segment used to derive the latent representations of these frames. Subsequently, a random search procedure (random search cv) is utilized to estimate the optimal number of clusters. K-Means clustering is now applied to these representations, with the frame closest to each cluster centroid selected as a keyframe. Experimental outcomes show that the suggested method increases the F-score of keyframe based summarization more than the existing methods. The proposed approach efficiently extracts keyframes from video data, with potential applications in video summarization and indexing