A Poodle or a Dog? Evaluating Automatic Image Annotation Using Human Descriptions at Different Levels of Granularity
Josiah Wang, Fei Yan, Ahmet Aker, Robert Gaizauskas · 2014
Different people may describe the same object in different ways, and at varied levels of granular-ity (“poodle”, “dog”, “pet ” or “animal”?) In this paper, we propose the idea of ‘granularity-aware ’ groupings where semantically related concepts are grouped across different levels of granularity to capture the variation in how different people describe the same image content. The idea is demonstrated in the task of automatic image annotation, where these semantic group-ings are used to alter the results of image annotation in a manner that affords different insights from its initial, category-independent rankings. The semantic groupings are also incorporated during evaluation against image descriptions written by humans. Our experiments show that se-mantic groupings result in image annotations that are more informative and flexible than without groupings, although being too flexible may result in image annotations that are less informative. 1