IMSearch: An Interactive Multimedia Video-Moment Search System
Duc-Tuan Luu, Duy-Ngoc Nguyen, Khanh-Linh Bui-Le, Vinh-Tiep Nguyen, Minh–Triet Tran · 2024
In recent years, video content has become increasingly popular due to the development of technologies, especially for recording devices. This explosion has driven the need for an advanced video-moment retrieval system that can accurately search for specific video segments matching the intentions of the user's queries. A prominent challenge in this field lies in the multimedia nature of the video data, which includes visual, auditory and even textual information. The accuracy and relevance of search results depend on how efficiently a retrieval system processes these types of metadata. In this paper, we introduce IMSearch, an interactive multimedia search system that can retrieve precise video-moment content by executing queries across various information types, such as text, audio, location of the objects, sketch, and human pose. Additionally, we implement a re-ranking mechanism based on user feedback to simultaneously improve and optimize the performance of our IMSearch system. We also explore two different vector libraries (FAISS and HNSW) for vector searching and report the retrieval accuracy and executed time to demonstrate our proposed paradigm.