Probabilistic Semantic Video Indexing
Milind Ramesh Naphade, Igor V. Kozintsev, Thomas S. Huang · 2000
We propose a novel probabilistic framework for semantic video in-dexing. We define probabilistic multimedia objects (multijects) to map low-level media features to high-level semantic labels. A graphical network of such multijects (multinet) captures scene con-text by discovering intra-frame as well as inter-frame dependency relations between the concepts. The main contribution is a novel application of a factor graph framework to model this network. We model relations between semantic concepts in terms of their co-occurrence as well as the temporal dependencies between these concepts within video shots. Using the sum-product algorithm [1] for approximate or exact inference in these factor graph multinets, we attempt to correct errors made during isolated concept detec-tion by forcing high-level constraints. This results in a significant improvement in the overall detection performance. 1