An efficient method of language identification using LVQ network
Han Xiao, Yu Lei, Kai Chen · 2008
This paper presents a new method to identify languages. A LVQ (learning vector quantization) network aimed at language identification is introduced. The presence of particular characters, words and the statistical information of word lengths are used as a feature vector. The new classification technique is faster than the conventional N-gram based classification approach, but it performs similarly in correct classification rate. In an identification experiment with 8 Roman alphabet languages, the LVQ network achieved 97.6% correct classification rate with 500 bytes, but it is five times faster than N-gram based approach.