Relation Extraction using Multiple Pre-Training Models in Biomedical Domain
Satoshi Hiai, Kazutaka Shimada, Taiki Watanabe, Akiva Miura, Tomoya Iwakura · 2021
The number of biomedical documents is increasing rapidly.Accordingly, a demand for extracting knowledge from large-scale biomedical texts is also increasing.BERT-based models are known for their high performance in various tasks.However, it is often computationally expensive.A high-end GPU environment is not available in many situations.To attain both high accuracy and fast extraction speed, we propose combinations of simpler pre-trained models.Our method outperforms the latest state-of-the-art model and BERT-based models on the GAD corpus.In addition, our method shows approximately three times faster extraction speed than the BERT-based models on the ChemProt corpus and reduces the memory size to one sixth of the BERT ones.