Use of Neural Machine Translation in Multimodal Translation
M. Sneha Nair, Sarvesh Tanwar, Sumit Badotra, Vinay Kukreja · 2023
Multimodal Neural Machine Translation (MNMT) is a type of Machine Translation that allows the translation of source language that contains various forms of information, such as images, video, audio, and signs, into the target language while maintaining the intended meaning of the source language. In MNMT, the use of deep learning techniques such as Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) are required to process different modalities and to generate an appropriate translation. The use of attention mechanisms, early and late fusion techniques, and hierarchical fusion have significantly improved the accuracy and fluency of translation. The performance of different MNMT systems can be evaluated in terms of various metrics such as BLEU score, accuracy, and fluency. MNMT has the potential to revolutionise the way we communicate by bridging the gap between different languages and cultures. The integration of MNMT in various industries, such as e-commerce, tourism, and healthcare, can bring significant benefits in terms of customer experience and accessibility. It states how Neural Machine Translation plays a significant role in processing different modalities in Multimodal Neural Machine Translation. More research work in this field can produce effective and innovative ideas to improve Multimodal Neural Machine Translation.