Cybersecurity Named Entity Recognition Based on Word-level Enhancement and Multi-task Learning
Yiqin Lu, Hong Qi, Jiancheng Qin, Jiarui Chen, Weiqiang Pan · 2023
At present, the situation of cybersecurity is becoming increasingly serious, and the study of Named Entity Recognition (NER) in the field of cybersecurity is helpful to automatically extract cybersecurity entities. It is of great significance for the subsequent analysis of cybersecurity. However, less study is done in the area of cybersecurity and the majority of the current research only applies to the general field. And for the dataset mixed with Chinese and English in the field of cybersecurity, the existing NER method is difficult to solve the problem of ambiguity in word boundaries. To solve this problem, this paper proposes a cybersecurity NER model based on word-level enhancement and multi-task learning on BERT-BiLSTM-Att-CRF (MTLWE). One the one hand, MTLWE obtains character-level feature vectors through the BERT pre-trained language model. In order to strengthen the word boundary feature information, the cybersecurity word embedding vector is obtained by Word2Vec algorithm. And the combination of these two vectors can enhance feature information. On the other hand, in the model, we configure the model to train the NER task and the Word Segmentation (WS) task alternately. In this way, the WS information is fused into the NER task and it produces the impact of multi-task learning. At the same time, adversarial training is used to filter the private independent information of WS task to ensure the purity of the common features of multi-task learning. Finally, the MTLWE model is experimented on the constructed cybersecurity dataset, and the F1 value reaches 65.22%. The results show that the MTLWE model shows better performance than other baseline models.