Implementation and Analysis of Different POS Tagger in Khasi Language
Risanlang Hynniewta, Arnab Kumar Maji, Sunita Warjri · Advances in IT standards and standardization research (AISSR) book series/Advances in IT standards and standardization research series · 2022
Part-of-speech tagging is a process of assigning each word of a sentence to a part of speech based on its context and definition. POS tagging is a prerequisite tool for many NLP tasks like word sense disambiguity, name entity information extraction, etc. Unfortunately, very little work has been done so far in this line for Khasi Language. The main difficulty lies with the unavailability of an annotated corpus. Hence, a small corpus is created which consists of 778 sentences with 34,873 words, out of which 3,942 are distinct words and a tagset of 52 tags. In this chapter, three methods for POS tagging, namely Brill's tagger, hidden Markov model (HMM)-based tagger, and bidirectional long short-term memory recurrent neural network (Bi-LSTM), have been implemented. Then a comparative analysis is performed, and it is observed that Bi-LSTM performs better in terms of accuracy.