Incorporation of question segregation procedures in visual question-answering models
Souvik Chowdhury, Badal Soni, Doli Phukan · International Journal of Computing Science and Mathematics · 2024
There are various open issues in visual question answering (VQA). One of them is sometimes a model can predict 'Yes' or 'No' as an answer, which is not relatable to the question and requires a descriptive answer, and vice versa. To solve this issue in the VQA domain, in this paper, a question segregation (QS) technique is incorporated to classify the questions into three types ('Yes/No', 'Other' and 'Number'). Then we successfully incorporated this technique with two of the VQA models, stacked attention networks (SAN) and modular co-attention network (MCAN). We evaluate the performance of the QS and SAN models on two datasets, VQA v.2 and CLEVR. We also studied and analysed the impact of question segregation on the performance of these two models on different datasets.