Automatic Song Translation for Tonal Languages
Fenfei Guo, Chen Zhang, Zhirui Zhang, Qixin He, Kejun Zhang, Jun Xie, Jordan Lee Boyd-Graber · Findings of the Association for Computational Linguistics: ACL 2022 · 2022
This paper develops automatic song translation (AST) for tonal languages and addresses the unique challenge of aligning words' tones with melody of a song in addition to conveying the original meaning.We propose three criteria for effective AST-preserving meaning, singability and intelligibility-and design metrics for these criteria.We develop a new benchmark for English-Mandarin song translation and develop an unsupervised AST system, Guided AliGnment for Automatic Song Translation (GagaST), which combines pre-training with three decoding constraints.Both automatic and human evaluations show GagaST successfully balances semantics and singability.