TransLLaMa: LLM-based Simultaneous Translation System
Roman Koshkin, Katsuhito Sudoh, Satoshi Nakamura · 2024
Decoder-only large language models (LLMs) have recently demonstrated impressive capabilities in text generation and reasoning.Nonetheless, they have limited applications in simultaneous machine translation (SiMT), currently dominated by encoder-decoder transformers.This study demonstrates that, after fine-tuning on a small dataset comprising causally aligned source and target sentence pairs, a pre-trained open-source LLM can control input segmentation directly by generating a special "wait" token.This obviates the need for a separate policy and enables the LLM to perform English-German and English-Russian SiMT tasks with BLEU scores that are comparable to those of specific state-ofthe-art baselines.We also evaluated closedsource models such as GPT-4, which displayed encouraging results in performing the SiMT task without prior training (zero-shot), indicating a promising avenue for enhancing future SiMT systems.The code is available at https://github.com/RomanKoshkin/transllama.