ArTST: Arabic Text and Speech Transformer
Hawau Olamide Toyin, Amirbek Djanibekov, Ajinkya C. Kulkarni, Hanan Aldarmaki · 2023
We present ArTST, a pre-trained Arabic text and speech transformer for supporting opensource speech technologies for the Arabic language.The model architecture follows the unified-modal framework, SpeechT5, that was recently released for English, and is focused on Modern Standard Arabic (MSA), with plans to extend the model for dialectal and codeswitched Arabic in future editions.We pretrained the model from scratch on MSA speech and text data, and fine-tuned it for the following tasks: Automatic Speech Recognition (ASR), Text-To-Speech synthesis (TTS), and spoken dialect identification.In our experiments comparing ArTST with SpeechT5, as well as with previously reported results in these tasks, ArTST performs on a par with or exceeding the current state-of-the-art in all three tasks.Moreover, we find that our pre-training is conducive for generalization, which is particularly evident in the low-resource TTS task.The pre-trained model as well as the fine-tuned ASR and TTS models are released for research use.