Advancing Prosthetic Vision in Education with Event-Driven Human Pose Estimation
Ming Li, Anran Meng, Xiaoming Chen, Chen Wang, Yuk Ying Chung · 2024
Motion and gestures play a crucial role in educational communication and learning activities, such as sign language learning, dance learning, and other learning activities. However, learners with visual impairments, including those with retinal degenerative diseases like retinitis pigmentosa and age-related macular degeneration, are deprived of the ability to visually observe and learn these activities, resulting in significant educational disadvantages. To address this challenge, we propose leveraging event cameras, which are emerging neuromorphic sensors capable of capturing high-temporal-resolution brightness changes, to perform human pose estimation for educational purposes. This approach not only aids in precise motion capture but also ensures the rapid processing of dynamic movements, enabling the creation of skeletal diagrams that effectively represent motion dynamics and can serve as inputs for visual prostheses of visually impaired learners. By utilizing event-driven human pose estimation, we can enhance the learning experience for visually impaired learners by providing them with a better understanding of motion and gestures. Our evaluation demonstrates that event-based human pose estimation significantly improves the accuracy and efficiency of motion and gesture recognition for educational purposes. As a direct outcome of this research, more equitable learning opportunities will be created for visually impaired learners. This achievement marks a significant step towards ensuring accessibility and educational equality for all, regardless of their visual abilities.