AGAIN: A Multimodal Human-Centric Event Retrieval System using dual image-to-text representations
Minh-Nam Tran, Tuan-An To, Viet-Nhat Thai, Thanh-Duy Cao, Trong-Tin Nguyen · 2023
Event retrieval from visual data presents a formidable challenge, requiring the identification of specific events from a pool of similar occurrences based on concise textual descriptions or a sequence of consecutive frames. The AI Challenge HCMC 2023 - Event Retrieval from Visual Data was a competition to encourage scientists to create interactive retrieval systems tailored for extensive Vietnamese video news databases. Participating teams promptly submitted precise event identifications derived from visual data or natural language descriptions. Therefore, we propose a lightweight and user-friendly yet robust event retrieval system, incorporating functionalities such as image-text matching, object detection, automatic speech recognition, and optical character recognition. This system enables users to search for events through natural language descriptions and relevant images. Notably, our team, named "Again," secured the second position on the public leaderboard of the competition using this system.