Bi-directional ConvLSTM-based interactive query focused video summarisation
Vasudha Tiwari, Charul Bhatnagar · International Journal of Wireless and Mobile Computing · 2024
Video summarisation presents the essence of a video in a compact form by extracting the most salient frames from it in a temporal sequence. Although, over time, video summarisation has gained much focus from the researchers, yet the need of personalised summaries that are based on user's intent still needs exploration. The authors propose an interactive, Query Focussed Video Summarisation (QFVS) approach which attempts to find those frames of a video that have maximum pertinence to user's text query. The proposed model consists of a Bi-directional ConvLSTM as an enhancement over ConvLSTM along with Resnet-50 for feature extraction. The input text query is matched with the predicted labels produced by the model and the frames with maximum similarity are selected for summary generation. The proposed method is evaluated on performance with previous state-of-art works and the results clearly demonstrate a significant improvement in the performance and prove the efficiency of the approach.