A BERT-Based Knowledge Selection Model for Bank Regulatory Reporting in Conversational Systems
Xiaoguo Wang, Jiaqi Cao · 2024
Knowledge selection is a natural language processing (NLP) task of selecting relevant knowledge spans from document text based on dialogue context in knowledge-grounded conversational systems. Based on the documents issued by the China Banking and Insurance Regulatory Commission, we build a conversational system for bank regulatory reporting through knowledge selection. In this paper, aiming at the documents’ characteristics of hierarchical structure, special terms, and long text, we propose the Bank Regulatory Reporting BERT (BR-BERT) for knowledge selection in the bank regulatory reporting conversational system. BR-BERT restricts the knowledge spans to a predefined span list. Based on the vector representation of dialogue history and document text encoded by BERT, predefined span position information was introduced into the model for classification to predict the start and end positions of knowledge spans. Compared with other models mentioned in the paper, the experimental results show that BR-BERT performs better such as exact match score (EM) and token-level F1 score (F1) on the task of knowledge selection for bank regulatory reporting text, verifying the effectiveness of the method.