Multiview Language Bias Reduction for Visual Question Answering
Pengju Li, Zhiyi Tan, Bing‐Kun Bao · IEEE Multimedia · 2022
Current visual question answering models overly rely on language bias and fail to understand visual information sufficiently. Many recent works concentrate on mitigating the intraquestion type bias (bias in the distribution of answers to a question type) without taking the interquestion type bias (bias in distribution between question types) into consideration, causing the model to ignore the tail question types. In addition, they neglect the overall distribution bias of the answer set, leading to the model only focusing on the head answers. In this article, we propose the Multi-View Language Bias Reduction (MVBR) method to solve these problems. For the interquestion type bias, we introduce the Inter-Question Type Bias (IQTB) module. IQTB exploits the question type distribution of the training set to determine the question type bias, which is used to generate weighting factors to reshape the loss of each question type into a balanced form. For the overall distribution bias of the answer set, we utilize the Decoupled Training (DT) module. The DT module penalizes the weights of each class in the classifier and forces their distribution to be balanced. Experimental results demonstrate that MVBR is effective on VQA-CP v2 and effectively improves performance on the mainstream models.