Two-Step Masked Language Model for Domain-Adapting Multi-Modal Task-Oriented Dialogue Systems
Youngjae Chang, Youngjoong Ko · IEEE/ACM Transactions on Audio Speech and Language Processing · 2023
The next generation of artificial intelligence (AI) is required to be capable of proper communication to enable eloquent interaction with human beings. Thus, AI models require powerful language understanding and generation capabilities. Therefore, designing an adequate dialogue-specific task is an important topic in dialogue research. In this paper, we improve our model that won in two out of four subtasks in SIMMC2.0 challenge using a dialogue-specific pre-training task, which learns the distinctive features of dialogues. In addition, we introduce an efficient method to pre-train the encoder-decoder transformer using an auxiliary task for the encoder. Experimental results indicate that superior performance is achieved compared to our previous model, revealing a new approach to improve task-oriented dialogue systems.