Concept-based Video Search with the PicSOM Multimedia Retrieval System
Ville Viitaniemi, Mats Sjöberg, Markus Koskela, Jorma T. Laaksonen · Aaltodoc (Aalto University) · 2010
In this report we describe the structure of the PicSOM multimedia retrieval system and elaborate on its automatic concept detection and video search subsystems.We evaluate several alternative techniques for implementing these two components of the PicSOM system in a comprehensive series of experiments employing the large-scale setups of the TRECVID video retrieval evaluation campaigns of 2008 and 2009.Based on the results of the experiments, we conclude that fusion-based shot-wise visual analysis together with N -gram temporal concept-wise post-processing make an efficient combination of techniques for automatic semantic concept detection from video material.It has previously been shown that semantic concepts are very beneficial for video search.Here we investigate the lexical and visualexample-based selection of concepts for search queries, concluding that both selection methods are successful in matching useful concepts.Finally, we show that the performance of the PicSOM system has improved since the TRECVID 2008 evaluation and now compares very well with the state-ofthe-art in concept detection and video search.