Toward Normalizing Romanized Gurumukhi Text from Social Media
Jagroop Kaur, JASWINDER PAL SINGH · Indian Journal of Science and Technology · 2015
Roman characters are used to write Indian language text on social media like facebook and twitter. Processing this text for NLP applications is not a trivial task. This text needs to be transliterated as well as conversion to canonical form. This paper discusses the various issues involved in normalizing such text in the domain of Punjabi Language. An algorithm is proposed to normalize Punjabi language text which is written using roman script. The proposed algorithm tries to find out all possible combinations and then filter using n-gram language model. Keywords: Text Normalization, Transliteration, Social Media Text