Voted Approach for Part of Speech Tagging in Bengali
Asif Ekbal, Mohammed Hasanuzzaman, Sivaji Bandyopadhyay · Institutional Repositories DataBase (IRDB) · 2009
Part of Speech (POS) tagging is the task of labeling each word in a sentence with its appropriate syntactic category called part of speech. POS tagging is a very important preprocessing task for language processing activities. In this paper, we report about our work on POS tagging for Bengali by combining different POS tagging systems using three weighted voting techniques. The individual POS taggers are based on Maximum Entropy (ME), Conditional Random Field (CRF) and Support Vector Machine (SVM) frameworks. The POS taggers use a tag set of 27 POS tags, defined for the Indian languages. The individual system makes use of the different contextual information of the words along with the variety of word3level features that are helpful in predicting the various POS classes. The POS tagger has been trained and tested with 57,341 and 35K tokens, respectively. It has been experimentally verified that the lexicon, named entity recognizer and different word suffixes are effective in handling the unknown word problems and improve the accuracy of the POS tagger significantly. Experimental results show the effectiveness of the proposed voted POS tagger with an accuracy of 92.35%, which is an improvement of 5.29% over the least performing ME based system and 2.23% over the best performing SVM based system.