Semantic Face Retrieval
Karthik Sridharan · 2006
The description of a face given by people is almost always semantic in nature using verbal terms such as “long face”, “thick lipped” or “blonde haired”. Existing Face Retrieval systems are mostly image based and hence do not capture these semantic features directly. Further since they perform image matching or other computationally intensive processes for retrieval of faces they are generally inefficient even on relatively small databases of face images. We propose a probabilistic and interactive semantic face retrieval system that retrieves face images based on verbal descriptions given by users. The system is also capable of prompting the user to provide information about facial features of the targeted face that best distinguishes the person from the top choices dynamically at any given time in the query session. The proposed system can supplement systems like Identikit (or other mug-shot retrieval systems) by providing an effective and efficient filter. Since the semantic query process is essentially a table search; the system is efficient and can operate in real-time. The proposed system automates the process of tagging face images with semantic labels which are then used for retrieving images that best fit the verbal description provided by the user. During enrollment, a critical step that needs to be performed for the automated semantic tagging of the face is facial feature localization and parameterization. We present two approaches for the task. First, in cases where mug-shot images are enrolled, primitive vision techniques and heuristics are used to locate and parameterize the facial features. Alternatively, in cases where unconstrained frontal face images are enrolled, a hybrid linear graphical model is proposed to model the relationship between the facial features and their locations based on a training set. This graphical model is then used to locate and parameterize facial feature extraction in a given face image. During the retrieval process Bayesian Inference is used. The system is interactive and prompts the user at each stage of the query process to provide the description of a facial feature that is most discriminative. In our experiments the target face image appeared 77.6% of the time within the top 5 and 90.4% of the time within the top 10 retrieved face images.