RS-MCQA: Multi-class Question Aware Visual Question Answering for Optical Remote Sensing Datasets
Hitul Desai, Debabrata Pal, Ankit Jha, Avik Hati, Biplab Banerjee · 2023
We tackle the problem of visual question answering (VQA) towards automated landcover information retrieval. Analysis of the remote sensing datasets involves simple classification problems to complex regression challenges of the widely diversified ground objects with varying shapes and sizes. Prior research carries low contextual information of the questions with the remotely sensed object images, impacting the generalization performance. This paper proposes a Global Residual DenseNet to exploit image feature reuse with a controlled growth rate in the channel dimension of convolution filters. Further, image and question embedding are stacked in interleave for a resonant coupling. Finally, we optimize multi-class question-aware decoders (RS-MCQA) using multi-class margin and binary cross-entropy loss for learning a robust semantic correlation between the image and the generic land-cover analysis questionaries. Experimental results on the benchmark RSVQA and RSIVQA datasets show the proposed method’s state-of-the-art performance in the context of remote sensing VQA task.