A Joint Intent Classification and Slot Filling Method Based on Knowledge-Distillation
Guangsheng Liu, Yipu Qin, Wenbin Luo, Ange Li, Xinbo Ai · 2023
With the increasing machine computing power, the deep learning model has a large number of parameters and is painful to deploy, research on knowledge distillation methods used to compress large models is receiving increasing attention. This paper proposes a new Knowledge-distillation (KD) approach that introduces contrast learning to enhance the generic representation of models and outstanding the differences between teacher and student. Furthermore, joint intent classification and word slot filling learning are also invoked to output the results of intent category and lexical sequence annotation, and a CRF module can optionally be added after the encoder. The experiment is based on the ATIS and SNIPS, and the results showed that our models achieved 98.84% and 99.14% on intent accuracy respectively. This study provides an effective solution for the intent recognition task, which can accelerate model inference and achieves an outstanding classification result.