Region proposal-based semantic matcher
Anastasiia Moiseenko, Yuri V. Vizilter, B. V. Vishnyakov, Vladimir S. Gorbatsevich, V. Pavlov Oleg · 2019
Unseen object detection problem is known as a semantic matching problem. Thus, a semantic matcher takes two images as an input – the request image and the test image. The request image represents an object class needed to be found on the test image. In this paper, we propose a new region proposal based semantic matcher. In our region based semantic matcher we use the same ideas as in R-CNN. Our Body CNN also generates proposals similar to classical Faster R-CNN, and Head-CNN compares proposals with a request descriptor, extracted from the request image. To extract features from the request image we use Request descriptor CNN. All three CNNs – Head, Body and Request descriptor are trained together, end-to-end for seen class object detection by request and then applied to both seen and unseen classes. We have trained and tested our CNN on Pascal VOC Dataset.