Memory-efficient NLLB-200: Language-specific Expert Pruning of a Massively Multilingual Machine Translation Model
Yeskendir Koishekenov, Alexandre Bérard, Vassilina Nikoulina · 2023
The recently released NLLB-200 is a set of multilingual Neural Machine Translation models that cover 202 languages.The largest model is based on a Mixture of Experts architecture and achieves SoTA results across many language pairs.It contains 54.5B parameters and requires at least four 32GB GPUs just for inference.In this work, we propose a pruning method that enables the removal of up to 80% of experts without further finetuning and with a negligible loss in translation quality, which makes it feasible to run the model on a single 32GB GPU.Further analysis suggests that our pruning metrics can identify language-specific experts.