Counting in Visual Question Answering: a concept detector based approach
Maaike de Boer, S. Reitsma, Klamer Schutte · International Conference on Multimedia Retrieval · 2016
Visual Question Answering is a field that combines vision techniques and natural language processing techniques. One of the most challenging question types in this field is counting, such as How many sheep are in this picture. In this paper, we focus on counting questions and improve upon the state-of-the-art method DPPnet. We train concept detectors on the MSCOCO dataset and use these detectors in addition to the pre-final layer from the original visual network. Additionally, we use a postprocessing technique to output the right type of answer to each type of question. Both the concept detectors, and the postprocessing slightly improve performance and is usable on current state-of-the-art methods.