RNNs with Attention for Machine Translation
M. Ümit Uyar · 2025
This chapter introduces a simple example for machine translation using a recurrent neural network (RNN) with the attention mechanism. In natural languages, placements of the subject, verb, preposition and tense information in a sentence significantly vary from one language to another. Because even the last words in a sentence may change its translation to another language, output generation does not start until all words in the input sentence have been processed by the RNN-Att. Another feature of machine translation is that the length of an input sentence may be different to its translation. Such characteristics necessitate a many-to-many RNN-Att architecture. Because natural languages tend to have large vocabularies and the size of the sample data may be in millions, RNN-Att architectures with multiple hidden and embedding layers are customary for machine translators.