CipherDAug: Ciphertext based Data Augmentation for Neural Machine Translation
Nishant Kambhatla, Logan Born, Anoop Sarkar · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · 2022
We propose a novel data-augmentation technique for neural machine translation based on ROT-k ciphertexts.ROT-k is a simple letter substitution cipher that replaces a letter in the plaintext with the kth letter after it in the alphabet.We first generate multiple ROT-k ciphertexts using different values of k for the plaintext which is the source side of the parallel data.We then leverage this enciphered training data along with the original parallel data via multi-source training to improve neural machine translation.Our method, CipherDAug, uses a co-regularization-inspired training procedure, requires no external data sources other than the original training data, and uses a standard Transformer to outperform strong data augmentation techniques on several datasets by a significant margin.This technique combines easily with existing approaches to data augmentation, and yields particularly strong results in low-resource settings.1