Spatiotemporal Modeling and Label Distribution Learning for Video Summarization
Wei-Ta Chu, Yu-Hsin Liu · 2019
For a video which content does not follow specific production rules, or without professional editing, at least two problems should be solved to generate a good video summary. First, the summarization system should jointly model visual content in the spatial domain and visual dynamics in the temporal domain. Second, the system should consider the inconsistency between users, i.e., different users may annotate the same video segment with different importance scores. In this paper, we present a video summarization system that models spatiotemporal information of video segments, and predicts the distribution of importance scores for each segment. Based on the estimated importance scores, video summaries are generated by picking the ones with higher scores. We especially demonstrate the effectiveness of label distribution learning based on two video benchmarks.