Demonstration of CameraMouseAI: A Head-Based Mouse-Control System for People with Severe Motor Disabilities
Farid Karimli, Hao Yu, Srishti Jain, Emmanuel S. Akosah, Margrit Betke, Wenxin Feng · 2024
We propose the mouse control system CameraMouseAI that includes real-time facial feature detection and new ways to map facial feature movements to mouse clicks. In addition to selecting items on the screen with the traditional dwell-time mechanism, users can perform mouse commands on the screen by opening their mouth or raising their eyebrows. The modular and open source design of CameraMouseAI enables developers to update the platform with new computer vision and machine learning models and extend its functionality. User experiments with the CameraMouseAI involved target selection, web browsing, and typing. A group of 12 adults without motor impairments and four adults with severe motor impairments were able to use the application to complete both target selection and web browsing tasks.