Human-in-the-Loop Refinement of Zero-Shot Object Detection for Domain-Specific Artwork Datasets
Alex Dalbøl, Christofer Meinecke, Stefan Jänicke · 2026
This work investigates the use of zero-shot object detection for analyzing Holocaust-related artworks, aiming to develop efficient and interpretable AI tools for cultural heritage professionals. Adopting an interdisciplinary approach that bridges computer vision, participatory design, and historical research, the study introduces CHORUS, a human-in-the-loop annotation framework that allows experts to review, correct, and extend AI-generated detections. The performance of two state-of-the-art zero-shot models, OWLv2 and GroundingDINO, is compared, emphasizing the influence of prompt engineering, label curation, and conservative detection behavior when working with sensitive visual material. The CHORUS prototype was developed in the context of the MEMORISE project and evaluated and tested with associated domain experts and heritage professionals. While current models remain limited in handling symbolic and interpretive content, the study demonstrates how human-guided AI can accelerate dataset creation and improve analytical accuracy in low-resource cultural heritage domains. The paper concludes with design recommendations and outlines future work toward a fully integrated annotation and training pipeline.