A lightweight biomedical named entity recognition with pre-trained model
Yunhe Gou, Jie Cao · 2023
Biomedical Named Entity Recognition (BioNER) is a specialized subfield of Named Entity Recognition (NER) that focuses on identifying and classifying named entities in biomedical and clinical texts. The goal of BioNER is to extract essential information, such as genes, proteins, diseases, drugs, et al., from scientific literature, electronic health records (EHRs), biomedical databases, and other biomedical text sources. The recognition and classification of these entities are crucial for various biomedical and healthcare-related tasks, including information retrieval, data integration, knowledge extraction, and drug discovery. Traditional BioNER methods typically involve rule-based approaches or machine learning algorithms, et al. These methods have been widely used before the advent of deep learning and transformer-based models. Bidirectional Encoder Representations from Transformers (BERT) is a groundbreaking transformer-based language model. It has revolutionized various natural language processing (NLP) tasks by capturing contextual information and obtain optimal results in multiple benchmarks. A lightweight BioNER optimized model from traditional BERT (LWNER) is proposed in this study, which can capture contextual information and its knowledge transfer from pre-training on large-scale text corpora without relying heavily on feature engineering and handcrafted rules. Fine-tuning BERT on biomedical-specific data helps adapt the model to the nuances and terminology of the biomedical domain. We conduct the method LWNER on BioCreative dataset, BC2GM, BC4CHEMD, BC5CDR, especially the chemical entity in BC5CDR achieve F1- score 91.3%. We construct an online web tool based on LWNER to identify the arbitrary text from scientific literatures for building knowledge graph.