Multi-grained Attention with Object-level Grounding for Visual Question Answering
Pingping Huang, Jianhui Huang, Yuqing Guo, Min Qiao, Yong Tian Zhu · 2019
Attention mechanisms are widely used in Visual Question Answering (VQA) to search for visual clues related to the question.Most approaches train attention models from a coarsegrained association between sentences and images, which tends to fail on small objects or uncommon concepts.To address this problem, this paper proposes a multi-grained attention method.It learns explicit wordobject correspondence by two types of wordlevel attention complementary to the sentenceimage association.Evaluated on the VQA benchmark, the multi-grained attention model achieves competitive performance with stateof-the-art models.And the visualized attention maps demonstrate that addition of objectlevel groundings leads to a better understanding of the images and locates the attended objects more precisely.