SeqPO-SiMT: Sequential Policy Optimization for Simultaneous Machine Translation
Ting Xu, Zhichao Huang, Jiankai Sun, Shanbo Cheng, Wai Pang Lam · 2025
We present Sequential Policy Optimization for Simultaneous Machine Translation (SeqPO-SiMT), a new policy optimization framework that defines the simultaneous machine translation (SiMT) task as a sequential decision making problem, incorporating a tailored reward to enhance translation quality while reducing latency.In contrast to popular Reinforcement Learning from Human Feedback (RLHF) methods, such as PPO and DPO, which are typically applied in single-step tasks, SeqPO-SiMT effectively tackles the multi-step SiMT task.This intuitive framework allows the SiMT LLMs to simulate and refine the SiMT process using a tailored reward.We conduct experiments on six datasets from diverse domains for En → Zh and Zh → En SiMT tasks, demonstrating that SeqPO-SiMT consistently achieves significantly higher translation quality with lower latency.In particular, SeqPO-SiMT outperforms the supervised fine-tuning (SFT) model by 1.13 ‡ points in COMET, while reducing the Average Lagging by 6.17 in the NEWSTEST2021 En → Zh dataset.While SiMT operates with far less context than offline translation, the SiMT results of SeqPO-SiMT on 7B LLM surprisingly rival the offline translation of high-performing LLMs, including Qwen-2.5-7B-Instruct and LLaMA-3-8B-Instruct. † Corresponding authors.* Work done when Ting Xu was interned at Bytedance.‡ Kocmi et al. (2024b) has indicated that an increase of 1 point in COMET represents a significant improvement.