Bridging the Gap: A Hybrid Approach to Medical Relation Extraction Using Pretrained Language Models and Traditional Machine Learning
Nesma AbdelAziz Hassan, Rania Ahmed Abdel Azeem Abul Seoud, Dina Salem · Journal of Advances in Information Technology · 2024
Feature engineering can be time-consuming and challenging, requiring expertise in Natural Language Processing (NLP) techniques and methods.The objective of this study was to explore the use of contextual word embeddings, specifically Bidirectional Encoder Representations from Transformers (BERT)-generated word embeddings, for biomedical relation extraction.The study utilized machine learning models, including Support Vector Machines, Random Forests, and K-nearest neighbor algorithms, to classify relationships between medical entities based on these embeddings.The attention mechanism of a pre-trained BERT model was also used to capture information related to the relationship between medical entities, leading to more advanced biomedical relation extraction.The performance of the machine learning classifiers was evaluated as classification models.The proposed approach outperformed the most recent state-ofthe-art model on two publicly available biomedical relation extraction datasets Chemical-Protein Interactions (ChemProt) and Drug-Drug Interactions (DDI), indicating that traditional machine-learning techniques can compete with recent advancements.Experiments on the ChemProt dataset show that the performance of the proposed model's F1-Score is 0.778 and on the DDI dataset, F1-Score is 0.815.This study has demonstrated the potential for using contextual word embeddings and machine learning models for biomedical relation extraction, without the need for extensive manual feature engineering.