Mencius: A Chinese Named Entity Recognizer Using the Maximum Entropy-based Hybrid Model
Richard Tzong‐Han Tsai, Shih-Hung Wu, Cheng‐Wei Lee, Cheng-Wei Shih, Wen−Lian Hsu · 2004
This paper presents a Chinese named entity recognizer (NER): Mencius. It aims to address Chinese NER problems by combining the advantages of rule-based and machine learning (ML) based NER systems. Rule-based NER systems can explicitly encode human comprehension and can be tuned conveniently, while ML-based systems are robust, portable and inexpensive to develop. Our hybrid system incorporates a rule-based knowledge representation and template-matching tool, InfoMap [1], into a maximum entropy (ME) framework. Named entities are represented in InfoMap as templates, which serve as ME features in Mencius. These features are edited manually and their weights are estimated by the ME framework according to the training data. To avoid the errors caused by word segmentation, we model the NER problem as a character-based tagging problem. In our experiments, Mencius outperforms both pure rule-based NER systems. The F-Measures of person names (PER), location names (LOC) and organization names (ORG) in the experiment are respectively 94.3%, 77.8% and 75.3%. We also compared the NER results with/without word segmentation and found slight differences. 1