Two-Step Training and Mixed Encoding-Decoding for Implementing a Generative Chatbot with a Small Dialogue Corpus
Jintae Kim, Hyeon-gu Lee, Harksoo Kim, Yeonsoo Lee, Young-Gil Kim · 2018
Generative chatbot models based on sequence-to-sequence networks can generate natural conversation interactions if a huge dialogue corpus is used as training data.However, except for a few languages such as English and Chinese, it remains difficult to collect a large dialogue corpus.To address this problem, we propose a chatbot model using a mixture of words and syllables as encoding-decoding units.In addition, we propose a two-step training method, involving pre-training using a large non-dialogue corpus and re-training using a small dialogue corpus.In our experiments, the mixture units were shown to help reduce out-of-vocabulary (OOV) problems.Moreover, the two-step training method was effective in reducing grammatical and semantical errors in responses when the chatbot was trained using a small dialogue corpus (533,997 sentence pairs).