Augmenting Kannada Educational Video with Indian Sign Language Captions Using Synthetic Animation
D. S. Jayalakshmi, Hrishikesh Salpekar, Raj H. K. Kiran, R. Rahul, Shobha Shobha · 2020 Fourth World Conference on Smart Trends in Systems, Security and Sustainability (WorldS4) · 2020
The project aims for a machine translation system for augmenting Kannada educational video by providing a translation to Indian Sign Language (ISL) using virtual animation approach along with the subtitles. The system accepts Kannada educational video clips as input and output is delivered as 3D character animation. The first phase approach of the proposed system makes use of SpeechRecognition like kaldi an open source for converting the input video files to their respective text which are used to generate the subtitles and these texts are further pre-processed in order to meet the sign language norms. The right SiGML file for these pre-processed words is picked from the database. These SiGML files consist of XML tags which correspond to the respective HamNoSys Notations. HamNoSys is an alphabetic system that mostly phonetically describes signs. A synthetic animation of a human in the tool in the URL APP, JA SiGML takes input as these SiGML files and generates signs of the respective pre-processed words. Synchronization of inputted video with the sign language and the subtitle generation is vital for the clear and clean flow. This system is Tested by professional sign language trainer/interpreter for the sign language generated by the avatar and is found effective. The accuracy of sign language shown was found out to be 63%.