TreeBERT: Advanced Representation Learning for Relation Extraction
Shashank Hebbar, Ying Xie, Jiang Zhong · 2023
As the COVID-19 pandemic has subsided, it remains crucial to analyze the vast amount of research produced during this period to advance our understanding of effective medical treatments and therapies. The wealth of biomedical literature generated during the pandemic presents a challenge for researchers who need to quickly assimilate new information. To tackle this issue, we introduce Tree-BERT, a novel representation learning language model designed for the automatic extraction of biomedical relationships from medical publications. Unlike traditional BERT models and their variants, which solely focus on the contextual relationship between words in input sentences, Tree-BERT also incorporates sentence structure information, enhancing word embeddings. Our experiments reveal that Tree-BERT outperforms alternative methods in biomedical relationship extraction. We will also demonstrate the model’s ability to identify new relationships from Covid-19 publications, showcasing its potential to contribute to the ongoing analysis of pandemic research and its broader applications in the field of bio-medicine.