DCU-UvA Multimodal MT System Report
Iacer Calixto, Desmond Elliott, Stella Frank · 2016
We present a doubly-attentive multimodal machine translation model.Our model learns to attend to source language and spatial-preserving CONV 5,4 visual features as separate attention mechanisms in a neural translation model.In image description translation experiments (Task 1), we find an improvement of 2.3 Meteor points compared to initialising the hidden state of the decoder with only the FC 7 features and 2.9 Meteor points compared to a text-only neural machine translation baseline, confirming the useful nature of attending to the CONV 5,4 features.