iRAG: Advancing RAG for Videos with an Incremental Approach
Md Adnan Arefeen, Biplob K. Debnath, Md Yusuf Sarwar Uddin, Srimat Chakradhar · 2024
Retrieval-augmented generation (RAG) systems combine the strengths of language generation and information retrieval to power many real-world applications like chatbots. Use of RAG for understanding of videos is appealing but there are two critical limitations. One-time, upfront conversion of all content in large corpus of videos into text descriptions entails high processing times. Also, not all information in the rich video data is typically captured in the text descriptions. Since user queries are not known apriori, developing a system for video to text conversion and interactive querying of video data is challenging.