Task-Aware Generative Score: A New Metric for Generated Images in Object Detection Task
Wenqiao Li, Qiang Tong, Na Yan, Tianyu Wu, Xiulei Liu · 2024
Current large-scale image generation models have gained increasing attention for realistic and creative generated images. However, existing generation metrics for evaluating generated images lack the exploration of object detection tasks. To address this issue, we introduce Task-Aware Generative Score (TAGS), which combines detector scores with natural image quality evaluation, focusing on both the natural quality of the image and its suitability for object detection tasks. Through the application of TAGS, we revealed that existing image generation models inadvertently introduce synthetic samples and scenes not present in the original dataset when employed as augmentation data for object detection tasks. This phenomenon significantly impairs model accuracy by creating distributional discrepancies between training and test data distributions. Therefore, we propose Reference-Guided Data Augmentation model (RGA) based on reference image guidance. It automatically selects reference images from the dataset and generates them based on the specified area of the image using the generative ability of the Stable Diffusion large model. Additionally, a two-stage allocation strategy is proposed to help the model learn more complete knowledge through two stages of training with different data. Our experiments show that TAGS has a higher correlation with downstream task results than other metrics. We further conducted experiments to demonstrate the powerful capabilities of RGA model and the two-stage allocation strategy.