THE IWSLT 2021 BUT SPEECH TRANSLATION SYSTEMS
Hari Krishna Vydana, Martin Karafiát, Lukáš Burget, Jan Černocký · 2021
The paper describes BUT's English to German offline speech translation (ST) systems developed for IWSLT2021.They are based on jointly trained Automatic Speech Recognition-Machine Translation models.Their performances is evaluated on MustC-Common test set.In this work, we study their efficiency from the perspective of having a large amount of separate ASR training data and MT training data, and a smaller amount of speechtranslation training data.Large amounts of ASR and MT training data are utilized for pretraining the ASR and MT models.Speechtranslation data is used to jointly optimize ASR-MT models by defining an end-to-end differentiable path from speech to translations.For this purpose, we use the internal continuous representations from the ASR-decoder as the input to MT module.We show that speech translation can be further improved by training the ASR-decoder jointly with the MT-module using large amount of text-only MT training data.We also show significant improvements by training an ASR module capable of generating punctuated text, rather than leaving the punctuation task to the MT module.