A Unified Knowledge Graph Augmentation Service for Boosting Domain-specific NLP Tasks
Ruiqing Ding, Xiao Han, Leye Wang · 2023
By focusing the pre-training process on domain-specific corpora, some domain-specific pre-trained language models (PLMs) have achieved state-of-the-art results.However, it is under-investigated to design a unified paradigm to inject domain knowledge in the PLM finetuning stage.We propose KnowledgeDA, a unified domain language model development service to enhance the task-specific training procedure with domain knowledge graphs.Given domain-specific task texts input, KnowledgeDA can automatically generate a domain-specific language model following three steps: (i) localize domain knowledge entities in texts via an embedding-similarity approach; (ii) generate augmented samples by retrieving replaceable domain entity pairs from two views of both knowledge graph and training data; (iii) select high-quality augmented samples for fine-tuning via confidence-based assessment.We implement a prototype of KnowledgeDA to learn language models for two domains, healthcare and software development.Experiments on domainspecific text classification and QA tasks verify the effectiveness and generalizability of KnowledgeDA.