Kannada Kali: A Smartphone Application for Evaluating Spoken Kannada Words and Detecting Mispronunciations Using Self Organizing Maps
Savitha Murthy, Ankit Anand, Avinash Kumar, Ajay Cholin, Ashwitha Shetty, Aditya D. Bhat, Akshay Venkatesh, Lingaraj Kothiwale, Viraj Kumar, Dinkar Sitaram · 2018
Computer Aided Pronunciation Training (CAPT) systems can assist people learning to speak new languages by detecting and correcting mispronunciations. "Kannada Kali" is a prototype Android application that leverages learners' increasing access to smartphones to evaluate the pronunciation of Kannada words and provide feedback using a cloud-based framework. A CAPT system typically uses an Automatic Speech Recognition (ASR) sub-system. For sufficient accuracy, ASR systems need to be trained using speech data from both native (L1) and non-native (L2) speakers. Since the latter type of data is particularly difficult to gather, we follow recent research efforts that seek to minimize the dependency on large speech corpora. We recorded 21 Kannada words (two to five syllables long) pronounced correctly by a Kannada teacher as templates, and 1169 samples of these words spoken by 19 native and non-native Kannada speakers aged 18 to 25 years. These samples were manually rated on a 5-point Likert scale by the Kannada teacher and used to train a neural network classifier for our application. "Kannada Kali" provides learners feedback that matches the teacher ratings with an accuracy of 86% on binary classification and 68% on multi-class classification. We also propose a novel approach for detecting mispronunciations using Self Organizing Maps (SOM) and report promising initial results.