Automated Morphological Segmentation and Evaluation
Uwe D. Reichel, Karl Weilhammer · 2004
In this paper we introduce (i) a new method for morphological segmentation of part of speech labelled German words and (ii) some measures related to the MDL principle for evaluation of morphological segmentations.The segmentation algorithm is capable to discover hierarchical structure and to retrieve morphemes not present as simplex forms in the data.It achieved 87 % recall and 98 % precision.Regarding MDL based evaluation, a linear combination of vocabulary size and size of reduced deterministic finite state automata matching exactly the segmentation outputs turned out to be an appropriate measure to rank segmentation models according to their quality.