Balancing and Contrasting Biased Samples for Debiased Visual Question Answering
Runlin Cao, Zhixin Li · 2023
The goal of Visual Question Answering (VQA) is to test the reasoning ability of an intelligent agent by evaluating visual and textual information. However, recent studies suggest that many VQA models may only capture the correlation between questions and answers in the dataset rather than demonstrating true reasoning ability. To address this issue, we propose a new training approach called Balancing and Contrasting Biased Samples for Debiased VQA (BC-VQA) to build a robust VQA model. In our approach, we first generate two types of negative samples to balance the biased data and use self-supervised auxiliary tasks to help the base VQA model overcome language priors. Our method does not require any additional annotations. We then filter out biased training samples, construct positive samples by eliminating spurious correlations in biased samples, and perform auxiliary training through contrastive learning. Our approach is straightforward to implement and compatible with various VQA backbones. The experimental results demonstrate that BC-VQA achieves higher accuracy on VQA-CP v2 compared to the current state-of-the-art approaches.