An interface for human-computer interaction based on face feature tracking in two dimensions
George C. Stockman, Vera Bakić · 2000
In this dissertation, a new approach to control a computer using head movements, gaze direction and face expressions is discussed. A workstation user is observed in a non-intrusive fashion. Algorithms to find a user's face and where the user is looking at the workstation display are described. The user can issue a command to the computer by making a certain facial expression. The following modules are described: (i) face detection that is based on a skin-color model and motion, (ii) face feature location that is based on the knowledge of the face geometry, (iii) gaze location detection that is based on an Artificial Neural Network mapping of face feature location to display location and a mapping of mouse movement to display movement, and (iv) selection that is done through face expressions. In a computer implementation, real-time rates are achieved, and between 10 and 30 frames per second can be processed. Essentially, a handless Human-Computer Interaction (HCI) is provided that can have various applications. Disabled people who cannot use their hands can greatly benefit from such an HCI. Other users can use the interface as a replacement for a conventional mouse. Finally, the interface can be adapted to monitor where the user is looking in time, and the results could be used for various analysis, e.g., in cognitive science for visual perception analysis, in marketing for display exploration analysis, etc. The results of a study that evaluated the usability of the head-eye input interface are presented in the final Chapter. These results show that the users were able to adjust to the new interface and perform a range of tasks common for current application programs. Among other tasks, the users successfully controlled Netscape: they browsed several WWW pages with an average of 5 erroneous selections and 8:35 minutes of time.