Fatality Killed the Cat or: BabelPic, a Multimodal Dataset for Non-Concrete Concepts
Agostina Calabrese, Michele Bevilacqua, Roberto Navigli · 2020
Thanks to the wealth of high-quality annotated images available in popular repositories such as ImageNet, multimodal language-vision research is in full bloom.However, events, feelings and many other kinds of concepts which can be visually grounded are not well represented in current datasets.Nevertheless, we would expect a wide-coverage language understanding system to be able to classify images depicting RECESS and REMORSE, not just CATS, DOGS and BRIDGES.We fill this gap by presenting BabelPic, a hand-labeled dataset built by cleaning the image-synset association found within the BabelNet Lexical Knowledge Base (LKB).BabelPic explicitly targets nonconcrete concepts, thus providing refreshing new data for the community.We also show that pre-trained language-vision systems can be used to further expand the resource by exploiting natural language knowledge available in the LKB.BabelPic is available for download at http://babelpic.org.