Chinese Electronic Medical Records Named Entity Recognition Based on RoBERTa-wwm and MultiHead Attention
Pengyu Hao, Lin Zhang · 2024
To address the low utilization of entity features and insufficient semantic representation in Chinese electronic medical records named entity recognition, a method based on RoBERTa-wwm (A Robustly Optimized BERT Pre-training Approach-Whole Word Masking) and multi-head attention is proposed. This approach incorporates the pre-trained language model RoBERTa-wwm for semantic representations enriched with prior knowledge. To record both local and global aspects of sequences, it makes use of multi-head self-attention layers and a Bidirectional Long Short-Term Memory (BiLSTM) network. Named entities are extracted through sequence decoding and tagging by a conditional random field (CRF) layer. Test findings on the “CCKS 2019” dataset show that this technique raises the F1 score to a higher level.