Sheffield MultiMT: Using Object Posterior Predictions for Multimodal Machine Translation
Pranava Madhyastha, Josiah Wang, Lucia Specia · 2017
This paper describes the University of Sheffield's submission to the WMT17 Multimodal Machine Translation shared task.We participated in Task 1 to develop an MT system to translate an image description from English to German and French, given its corresponding image.Our proposed systems are based on the state-of-the-art Neural Machine Translation approach.We investigate the effect of replacing the commonly-used image embeddings with an estimated posterior probability prediction for 1,000 object categories in the images.Recent work (Wu et al., 2016;You et al., 2016) exploits explicit, higher-level semantic representation of images for the tasks of image captioning and visual question answering.Instead of feeding