Session 1 Keynote DICTA 2009 Web Video Summarization and Retrieval
Tat‐Seng Chua · 2009
With the proliferation of text, images and videos on the Web, users are now able to find multimedia answers to almost any questions. Meanwhile, they are also bewildered by the huge amount of information routinely presented to them. This talk presents our research on finding video answers to two types of questions - the how-to and topic-based questions. For answering how-to type questions, the focus is on returning precise answers when available. This is accomplished in two steps by first performing the recall-driven video search, which aims to increase the coverage of question by supplementing it with other similar textual questions found on the Web; follow by precision-based video ranking, which performs visual, opinion and video redundancy analysis to locate the most relevant video answers from YouTube. For the topic-based questions which tend to return a large ranked list of diverse videos, the focus is on summarizing the results in order to present an overview to the users. Our multi-video summarization framework first finds key-shots through near-duplicate video analysis and ranking; extracts semantic tags associated with key shots through representativeness and descriptiveness analysis with random walk; and formulates summary through an optimization procedure that takes into account of users' preference and time limit. We conduct user studies on a wide variety of questions for over one hundred hours of videos crawled from YouTube. The evaluation demonstrates the feasibility and effectiveness of our proposed solutions. The talk also discusses directions for future research.