Relation Extraction with BERT-based Pre-trained Model
Haitao Yu, Yi Cao, Gang Cheng, Ping Xie, Yang Yang, Peng Yu · 2020
Distant supervision relation extraction is an effective method to extract the real relation between entities from unstructured corpus. However, affected by the hypothesis of distant supervision mechanism, relation extraction model often faces the disturbance of mislabeled data and noise samples. In order to alleviate the above problems and improve the performance, we propose a relation extraction framework based on Bert-based pre-trained models, Bert for Relation Extraction (BRE). BRE uses BERT as feature extractor and loads pre-trained parameters for fine-tuning. It integrates external semantic knowledge with entity relation knowledge in specific tasks to improve the performance of classifier. In addition, we designed position enhanced CNN module and time-decay selective attention mechanism for BRE to bridge the semantic gap between external knowledge and relation knowledge, and alleviate the problem of mislabeling and noise in the multi-instance learning mode. We conducted experiments on NYT-10 and GIDS datasets, and the results show that BRE achieves the best performance.