Two-Way Malaysian Sign Language Communication System for Inclusive Education
Veron Zhen Liang Hii, Aaron Ken Kiat Lo, Ida Pei Xin Lee, Alec Vince Gonzales Abuan, Sue Han Lee, Patrick Then · 2024
Inclusive education aims to create equal learning opportunities for all, but significant gaps still exist, particularly for the deaf community. This paper addresses these gaps by proposing a new educational platform that integrates cutting-edge technology to improve accessibility and engagement for deaf learners. Our solution introduces an AI-powered two-way sign language communication system specifically designed for integration into classrooms. With an avatar-based approach, the system focuses on simple technology transfer for Sign Language Production (SLP) and utilises advanced deep learning methods for Sign Language Recognition (SLR). This enables seamless and effective communication between deaf students and educators. To our knowledge, this is the first comprehensive approach to digital inclusion in inclusive learning that primarily addresses the specific needs of the deaf community. As part of our initiative, we have created the first Malaysian Sign Language (BIM) education dataset to serve as a benchmark in this area. A new user testing framework has also been developed to quantify the effectiveness of the system in an educational context. The results of the survey emphasise the critical importance and necessity of such an educational platform.