DBMAT: Research on Chinese Named Entity Recognition Using the Dilated Bidirectional Multi-layer Attentive Transformer Fusion Model
Tingjie Yu, Liefa Liao, Yiguo Yang, Wuyou Xia, Zhenyuan Zou · 2023
In the realm of Chinese Named Entity Recognition tasks, conventional models have frequently fallen short in adequately addressing linguistic features and recognizing the essential role of context. To address this challenge, our research presents a unique Chinese Named Entity Recognition model, referred to as the LERT-DBMAT-CRF model. Initially, the LERT pre-trained language model incorporates language-informed pretraining strategies to enrich the semantic attributes in textual data. Subsequently, we apply the DBMAT module, unifying bidirectional Long Short-Term Memory networks with residual dilated convolutional networks, coordinated through a multi-head additive attention mechanism. This approach enhances feature extraction by employing the Exponential Linear Unit function, thus enhancing the model's capacity to capture temporal and spatial information relevant to semantic features. Lastly, a Conditional Random Field layer is introduced to exploit contextual information for label prediction. Experimental results demonstrate the exceptional model performance, achieving impressive F1 scores of 97.38% and 96.55% on the Resume dataset and the MSRA dataset, respectively, surpassing the performance of current mainstream models.