Real-Time Augmented Reality Visual-Captions for Deaf and Hard-of-Hearing Children in Classrooms
Jingya Li · 2023
Deaf and hard-of-hearing (DHH) children experience difficulties in mainstream classrooms since they cannot access audio information effectively. Although tools like captions are developed specifically to assist DHH individuals, primary school children have trouble to read or understand the text. To address this challenge, this paper develops an AR-based visual-captions, which will transfer the speech to text and images in real-time and display the information around the teacher in classrooms. The AR visual-captions aims to help DHH children better receive information and enhance their classroom experience. We describe the concept, design and implementation of our prototype, and discuss future research directions.