Multi-Head Attention Transformer for Text2Text Translation
Shivakumar B, Upputuri Someswara Sandeep, Varshini V, P. Kumaran · Procedia Computer Science · 2025
Multi-head attention within transformer models enables translation systems to concentrate on multiple aspects of the input sequence at the same time, capturing complex contextual dependencies and subtleties in meaning. This paper explores the application of transformer models for Text2Text English to Tamil translation, particularly emphasizing their ability to handle the rich morphological structure of Tamil. We conduct a comparative analysis involving several translation systems, including MTIL-2017, CICT-2019 Hindi translators, the Amirtha Malayalam translator, our own transformer-based model, Bing Translator, and Google Translator. Using the BLEU metric for evaluation, we demonstrate the transformer model’s superior performance in generating fluent and contextually accurate translations. The results highlight the potential of avant-garde neural architectures in enhancing translation for low-resource languages.