Beyond Semantic Search: What You Observe May Not Be What You Think.
Chong‐Wah Ngo, Yu–Gang Jiang, Xiao-Yong Wei, Wan‐Lei Zhao, Feng Wang, Xiao Ying Wu, Hung‐Khoon Tan · 2008
This paper presents our approaches and results of the four TRECVID 2008 tasks we participated in: high-level feature extraction, automatic video search, video copy detection, and rushes summarization. In high-level feature extraction, we jointly submitted our results with Columbia University. The four runs submitted through CityU aim to explore context-based concept fusion by modeling inter-concept relationship. The relationship is modeled not based on semantic reasoning, but by observing how concepts correlate to each other, either directly or indirectly, in LSCOM common annotation [1]. An observability space (OS) [2] is thus built on top of LSCOM [1] and VIREO-374 [3] for performing concept fusion. Since 19 of the 20 concepts evaluated this year appeared in VIREO-374, we apply OS to re-rank the results of both old models from VIREO-374 and new models from a joint baseline submission with Columbia.- A CityU-HK1: re-rank A CU-run5 using OS – both positive and negative correlated concepts are used.- A CityU-HK2: re-rank A CU-run5 using OS – only positive correlated concepts are used.- A CityU-HK3: re-rank old models from VIREO-374 using OS – both positive and negative correlated concepts are used.- A CityU-HK4: re-rank old models from VIREO-374 using OS – only positive correlated concepts are used. In automatic search, we focus on concept-based video search. The search is beyond semantic reasoning, where we consider the fusion of detectors using concept semantics, co-occurrence, diversity, and detector robustness. Two runs are submitted based on the works in [2, 4].- F A 2 CityUHK1 1: multi-modality fusion of concept-based search (Run-2), query example based search (Run-4 and Run-5), and text baseline (Run-6).- F A 2 CityUHK2 2: concept-based search by fusing semantics, observability, reliability and diversity of concept detectors [2].- F A 2 CityUHK3 3: concept-based search using semantics reasoning [4, 5].- F A 2 CityUHK4 4: query-by-example – using VIREO-374 detection scores as features.- F A 2 CityUHK5 5: query-by-example – using motion histograms as features.