American Sign Language Recognition and Translation Feasibility Study
K Brady, M.S. Brandstein, Jennifer Melot, Youngjune Gwon, Jacob M. Williams, Elizabeth Salesky, Michael T. Chan, Parsa Khorrami, Nicolas Malyska · ePrints Soton (University of Southampton) · 2018
The development of a system for automatically and robustly translating between American SignLanguage (ASL) and spoken English in real time on mobile devices holds the promise of enabling naturaland spontaneous communication between Deaf ASL signers and English speakers anywhere and anytime.One key component of such a system is the automatic recognition of ASL signs, which is an active area ofresearch in the academic community. A number of other system challenges remain in order to supportdeploying this technology on mobile devices that include addressing compute limitations and recognition robustness for acquired signals that are highly variable (e.g., sign variation, apparent pose angle) and poorly matched to existing training corpora. While several commercial companies have pursued the development of mobile translation systems, none of them has successfully commercialized such a system to date. This report investigates the technical feasibility of performing real-time translation between ASL and spoken English on mobile devices. An important aspect of this investigation is the identification of the key technical challenges in developing such a system and the development of a roadmap for addressing these challenges. In order to support the feasibility study detailed in this report, an extensive literature search has been completed, a number of rigorous experiments have been performed to characterize state of the art performance, and a prototype system has been developed.