Structure-to-Text Generation with Self-Training, Acceptability Classifiers and Context-Conditioning for the GEM Shared Task
Shreyan Bakshi, Soumya Batra, Peyman Heidari, Ankit Arun, Shashank Sharma B. C. Jain, Michael White · 2021
We explore the use of self-training and acceptability classifiers with pre-trained models for natural language generation in structure-totext settings using three GEM datasets (E2E, WebNLG-en, Schema-Guided Dialog).With the Schema-Guided Dialog dataset, we also experiment with including multiple turns of context in the input.We find that self-training with reconstruction matching along with acceptability classifier filtering can improve semantic correctness, though gains are limited in the full-data setting.With context-conditioning, we find that including multiple turns in the context encourages the model to align with the user's word and phrasing choices as well as to generate more self-consistent responses.In future versions of the GEM challenge, we encourage the inclusion of few-shot tracks to encourage research on data efficiency.