Ground truth for training and evaluation of automatic main subject detection
Stephen P. Etz, Jiebo Luo · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · 2000
A consumer photograph, or snapshot, is a medium for conveying to a viewer, one's interest in one or more main subjects. A methodology is presented for collecting ground truth data useful for training and evaluating algorithms designed to automatically detect the main subject of a consumer photograph. For a database of 100 images, 16 observers provided polygonal approximations to the image areas that comprise the main subject. Results from all observer are combined to form a truth image that is considered the ideal result of a main subject detector and is analyzed to determine features for main subject detection (MSD). The collected ground truth shows substantial agreement among third-party observers. It also supports conventional wisdom regarding the likely locations of main subjects and the value of 'people' detection as a cue for main subject detection. Training data is created from the truth images for an MSD framework involving image segmentation, feature detection, and probabilistic reasoning. A proposed method for generating region-based training data can be used to retrain a reasoning engine as segmentation algorithms improve, without further observer involvement. Although the subject matter for consumer photographs ranges from sweeping landscapes to close portraits, identification of the main subject is a meaningful task.