Knowledge Distillation for Intent Detection and Slot Filling
Chenrui Ma, Yuhua Deng, Xiang Meng, Lining Yang · 2023
Intent detection and slot filling are two fundamental tasks in the spoken language understanding field (SLU). SLU is used to detect the user's intention and fill slots so that the system can correctly determine user's intention and react in a way that meets their demands. At the same time, the two tasks are highly dependent. To improve the accuracy of intention detection and slot filling, the Stack-Propagation model is adopted in this paper, in which intention information is directly used as input parameter for slot filling layers, so as to utilize semantic information and accuracy. Token-level intent detection is applied to effectively mitigate error propagation. To further improve the framework's efficiency, this paper pretreats the training set. Attain the POS tagging results of the word with slot 0 randomly and removes words with the same tagging in the training set. Based on the Stanford Core NLP package, we carry out POS tagging to achieve knowledge distillation. We conducted experiments on two benchmarks, ATIS and SNIPS. Finally, our optimized framework equipped with knowledge distillation shows a faster and more accurate performance.