Morphological Segmentation with LSTM Neural Networks for Tigrinya
Yemane Tedla, Kazuhide Yamamoto · International Journal on Natural Language Computing · 2018
Morphological segmentation is a fundamental task in language processing.Some languages, such as Arabic and Tigrinya, have words packed with very rich morphological information.Therefore, unpacking this information becomes a necessary task for many downstream natural language processing tasks.This paper presents the first morphological segmentation research for Tigrinya.We constructed a new morphologically segmented corpus with about 45,127 manually segmented tokens.Conditional random fields (CRF) and window-based long short-term memory (LSTM) neural networks were employed separately to develop our boundary detection models.We applied language-independent character and substring features for the CRF and character embeddings for the LSTM networks.Experiments were performed with four variants of the Begin-Inside-Outside (BIO) chunk annotation scheme.We achieved 94.67% F1 score using bidirectional LSTMs with window approach to morpheme boundary detection.