Real-Time Boundary Detection for Continuous Arabic Sign Language Translation
Nour Albasmy, Milad Michel Ghantous · 2024
Sign language is crucial for the deaf community but remains unfamiliar to many. While Static Sign Language Recognition (SSLR) and Isolated Sign Language Recognition (ISLR) have advanced, Continuous Sign Language Recognition (CSLR) still faces challenges, particularly in identifying word boundaries. This study introduces the Multi-FILSTM system, which uses Long Short-Term Memory (LSTM) networks to develop Fixed Input-Length LSTM (FILSTM) models for Arabic CSLR. The system analyzes frame sequences to detect high-confidence intervals for word boundary detection. A two-stage evaluation was conducted: first, FILSTM models were assessed; then, the combined models' real-time performance was evaluated based on CPU utilization, latency, and Continuous Sentence Recognition Accuracy (CSRA). The best-performing model achieved 9.10% CPU utilization, 267 ms latency, and a Word Recognition Rate (WRR) of 100%. The Multi-FILSTM shows significant potential to enhance CSLR for assistive technologies.