Named Entity Recognition via Interlayer Attention Residual LSTM
Gang Yang, Hongzhe Xu · 2022 International Joint Conference on Neural Networks (IJCNN) · 2022
As a type of powerful technique in neural networks, attention mechanisms have been successfully applied to many state-of-the-art models for NLP tasks. In this paper, we present an interlayer attention mechanism and introduce it to the stacked residual LSTM network for named entity recognition (NER). Compared to the conventional residual structure which adds the features from all the previous layers directly, the interlayer attention mechanism allows the model to focus on the most relevant features by assigning different weights to the features from different layers. Experimental results show that our model achieves state-of-the-art performance on both English and Chinese datasets, which demonstrates that the proposed interlayer attention mechanism can improve the performance of stacked residual LSTM network significantly.