MAGIC-VQA: Multimodal And Grounded Inference with Commonsense Knowledge for Visual Question Answering
Shuo Yang, Cheon Goo Han, Siwen Luo, Eduard H. Hovy · 2025
Visual Question Answering (VQA) requires reasoning across visual and textual modalities, yet Large Vision-Language Models (LVLMs) often lack integrated commonsense knowledge, limiting their robustness in real-world scenarios.To address this, we introduce MAGIC-VQA, a novel framework that enhances VQA by systematically integrating commonsense knowledge with LVLMs.MAGIC-VQA employs a three-stage process: (1) Explicit Knowledge Integration from external sources, (2) By-Type Post-Processing for contextual refinement, and (3) Implicit Knowledge Augmentation using a Graph Neural Network (GNN) for structured reasoning.While GNNs bring greater depth to structured inference, they enable superior relational inference beyond LVLMs.MAGIC-VQA bridges a key gap by unifying commonsensse knowledge with LVLMdriven reasoning, eliminating the need for extensive pre-training or complex prompt tuning.Our framework achieves state-of-the-art performance on benchmark datasets, significantly improving commonsense reasoning in VQA.Our implementation is open-sourced on GitHub 1