Exact Hard Monotonic Attention for Character-Level Transduction
Shijie Wu, Ryan Cotterell · 2019
Many common character-level, string-tostring transduction tasks, e.g.graphemeto-phoneme conversion and morphological inflection, consist almost exclusively of monotonic transduction.Neural sequence-tosequence models with soft attention, which are non-monotonic, often outperform popular monotonic models.In this work, we ask the following question: Is monotonicity really a helpful inductive bias in these tasks?We develop a hard attention sequence-to-sequence model that enforces strict monotonicity and learns a latent alignment jointly while learning to transduce.With the help of dynamic programming, we are able to compute the exact marginalization over all monotonic alignments.Our models achieve state-of-the-art performance on morphological inflection.Furthermore, we find strong performance on two other character-level transduction tasks.Code is available at https://github.com/ shijie-wu/neural-transducer.