Multi-task Label-wise Transformer for Chinese Named Entity Recognition
Xuelei Wang, Xirong Xu, Degen Huang, Ting Zhang · ACM Transactions on Asian and Low-Resource Language Information Processing · 2023
Benefiting from the improvement of positional encoding and the introduction of lexical knowledge, Transformer has achieved superior performance than the prevailing BiLSTM-based models in named entity recognition (NER) task. However, existing Transformer-based models for Chinese NER pay less attention to the information captured by the bottom layers of Transformer and the significance of representation subspace where each head of Transformer is projected. In this article, we propose M ulti- T ask L abel- W ise T ransformer (MTLWT). From a global perspective, we assign entity boundary prediction (EBP) and entity type prediction (ETP) tasks to the first two layers. In this way, we stimulate lower layers to participate more in constructing character representation. Besides, in each multi-head self-attention (MHSA) layer, we provide a specific focus for each individual head, making the head project into a significant subspace. Experiments on four datasets from different domains show that our proposed model achieves comparable performance with other state-of-the-art models. In particular, MTLWT outperforms the other frameworks without external knowledge on all the datasets.