Using face and object detection to quantify looks during social interactions
Shengyao Guo, Eric Ho, Yalun Zheng, Qiming Chen, Vivian Meng, John Cao, Si Wu, Leanne Chukoskie, Pamela C. Cosman · 2018
Quantifying gaze is important in various realms, such as evaluating atypical social looking behavior in autism spectrum disorder. This paper reports on a system that uses eye-tracking glasses and object/face detection to quantify looks. The algorithms use Viola-Jones face detection with feature point tracking and Faster-RCNN object detection trained for three objects, followed by a runlength algorithm to declare the start and end of looks. Results are presented in terms of bounding box overlap and accuracy of looks compared to a manual ground truth. The system can be useful for quantifying gaze behavior during dynamic social interactions.