Monolingual Adapters for Zero-Shot Neural Machine Translation
Jerin Philip, Alexandre Bérard, Matthias Gallé, Laurent Besacier · 2020
We propose a novel adapter layer formalism for adapting multilingual models.They are more parameter-efficient than existing adapter layers while obtaining as good or better performance.The layers are specific to one language (as opposed to bilingual adapters) allowing to compose them and generalize to unseen language-pairs.In this zero-shot setting, they obtain a median improvement of +2.77BLEU points over a strong 20-language multilingual Transformer baseline trained on TED talks.