Focused Evaluation for Image Description with Binary Forced-Choice Tasks
Micah Hodosh, Julia Hockenmaier · 2016
Current evaluation metrics for image description may be too coarse.We therefore propose a series of binary forced-choice tasks that each focus on a different aspect of the captions.We evaluate a number of different off-the-shelf image description systems.Our results indicate strengths and shortcomings of both generation and ranking based approaches.