Aggregating Machine Learning and Rule Based Heuristics for Named Entity Recognition
Karthik Gali, Harshit Surana, Ashwini Vaidya, Praneeth M Shishtla, Dipti Misra Sharma · International Joint Conference on Natural Language Processing · 2008
This paper, submitted as an entry for the NERSSEAL-2008 shared task, describes a system build for Named Entity Recognition for South and South East Asian Languages. Our paper combines machine learning techniques with language specific heuristics to model the problem of NER for Indian languages. The system has been tested on five languages: Telugu, Hindi, Bengali, Urdu and Oriya. It uses CRF (Conditional Random Fields) based machine learning, followed by post processing which involves using some heuristics or rules. The system is specifically tuned for Hindi and Telugu, we also report the results for the other four languages.