Unknown language rejection in language identification system
Hingkeung Kwan, K. Hirose · 2002
The number of languages in the world is much larger than the number of target languages that current language identification systems can handle. Therefore, we propose the use of a multilayer perceptron neural network as a means to prevent those unknown language inputs from being misidentified as one of the target languages. We consider not only the target language identification rate but also the unknown language rejection rate. The results reveal that, with the use of a phonemic unigram as the input feature to the neural network, a target language identification rate of 93.5% can be achieved for three languages. By varying the thresholds of the outputs, a good unknown language rejection rate can also be obtained at the expense of a lower identification rate.