Morphological analysis and decomposition for Arabic speech-to-text systems
Frank Diehl, Mark Gales, Marcus Tomalin, P. C. Woodland · 2009
Language modelling for a morphologically complex language such as Arabic is a challenging task. Its agglutinative structure re-sults in data sparsity problems and high out-of-vocabulary rates. In this work these problems are tackled by applying the MADA tools to the Arabic text. In addition to morphological decompo-sition, MADA performs context-dependent stem-normalisation. Thus, if word-level system combination, or scoring, is required this normalisation must be reversed. To address this, a novel context-sensitive method for morpheme-to-word conversion is introduced. The performance of the MADA decomposed sys-tem was evaluated on an Arabic broadcast transcription task. The MADA-based system out-performed the word-based system, with both the morphological decomposition and stem normalisa-tion being found to be important.