Overview of the ImageCLEF 2013 Personal Photo Retrieval Subtask.
David Zellhöfer · CLEF (Working Notes) · 2013
The subtask assesses the retrieval eectiveness in dierent retrieval usage scenarios in a personal photo collection and with dier- ent user groups. That is, the subtask reveals whether a tested algorithm is stable in terms of eectiveness for 7 dierent user groups. This per- spective on retrieval performance evaluation separates the 2013 version of the subtask from its pilot phase although it relies on the same data set. The data set has been sampled from 19 layperson photographers and consists of 5,555 unprocessed digital photographs. To solve the subtask, the participants are asked to retrieve the 100 best matching documents for 74 sample information needs that consist of visual concepts and events. Each sample information need is modeled by at most one query-by-example document and up to three to browsed documents. The best performing groups, ISI and DBIS, used visual low-level features and metadata to solve the task. The current best-placed run achieves a nDCG at 20 of 0.7427 for the average user group using relevance feedback and all available modalities, i.e., visual data and metadata such as Exif or GPS information. Regarding the stability, roughly 50% of the submitted runs perform equally well over all user groups.