Towards End-to-End In-Image Neural Machine Translation
Elman Mansimov, Mitchell E. Stern, Mia Chen, Orhan Fırat, Jakob Uszkoreit, Puneet Kumar Jain · 2020
In this paper, we offer a preliminary investigation into the task of in-image machine translation: transforming an image containing text in one language into an image containing the same text in another language.We propose an end-to-end neural model for this task inspired by recent approaches to neural machine translation, and demonstrate promising initial results based purely on pixel-level supervision.We then offer a quantitative and qualitative evaluation of our system outputs and discuss some common failure modes.Finally, we conclude with directions for future work.