Multimodal Approach for Summarizing and Indexing News Video
Jae-Gon Kim Kim, Hyun Sung Chang, Young-tae Kim Kim, Kyeongok Kang, Munchurl Kim Kim, Jinwoong Kim Kim, Hyung‐Myung Kim · ETRI Journal · 2002
A video summary abstracts the gist from an entire video and also enables efficient access to the desired content. In this paper, we propose a novel method for summarizing news video based on multimodal analysis of the content. The proposed method exploits the closed caption data to locate semantically meaningful highlights in a news video and speech signals in an audio stream to align the closed caption data with the video in a time-line. Then, the detected highlights are described using MPEG-7 Summarization Description Scheme, which allows efficient browsing of the content through such functionalities as multi-level abstracts and navigation guidance. Multimodal search and retrieval are also within the proposed framework. By indexing synchronized closed caption data, the video clips are searchable by inputting a text query. Intensive experiments with prototypical systems are presented to demonstrate the validity and reliability of the proposed method in real applications.