Neural Named Entity Recognition Using a Self-Attention Mechanism
Andrej Žukov-Gregorič, Yoram Bachrach, Pasha Minkovsky, Sam Coope, Bogdan Maksak · 2017
We propose a novel supervised approach for text tagging and multi-label text classification based on a multi-head encoder-decoder neural network architecture. Our method predicts which subset of possible tags best matches an input text. It efficiently spends computational resources, exploiting dependencies between tags by encoding an input text into a compact representation which is then passed to multiple decoder classifier heads. We test our architecture on a Twitter hashtag prediction task, comparing it to a baseline model with multiple feedforward networks and a baseline model with multiple recurrent neural networks with GRU cells. We show that our approach achieves a significantly better performance than baselines with an equivalent number of parameters.