Enhanced Language Representation with Label Knowledge for Span Extraction
Yang Pan, Xin Cong, Zhenyu Sun, Xingwu Liu · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · 2021
Span extraction, aiming to extract text spans (such as words or phrases) from plain texts, is a fundamental process in Information Extraction.Recent works introduce the label knowledge to enhance the text representation by formalizing the span extraction task into a question answering problem (QA Formalization), which achieves state-of-the-art performance.However, QA Formalization does not fully exploit the label knowledge and suffers from low efficiency in training/inference.To address those problems, we introduce a new paradigm to integrate label knowledge and further propose a novel model to explicitly and efficiently integrate label knowledge into text representations.Specifically, it encodes texts and label annotations independently and then integrates label knowledge into text representation with an elaborate-designed semantics fusion module.We conduct extensive experiments on three typical span extraction tasks: flat NER, nested NER, and event detection.The empirical results show that 1) our method achieves state-of-the-art performance on four benchmarks, and 2) reduces training time and inference time by 76% and 77% on average, respectively, compared with the QA Formalization paradigm.Our code and data are available at https://github.com/ Akeepers/LEAR.