The Case for a Single Model that can Both Generate Continuations and Fill-in-the-Blank
Daphne Ippolito, Liam D. Dugan, Emily Reif, Ann Yuan, Andy Coenen, Chris Callison-Burch · Findings of the Association for Computational Linguistics: NAACL 2022 · 2022
The task of inserting text into a specified position in a passage, known as fill in the blank (FITB), is useful for a variety of applications where writers interact with a natural language generation (NLG) system to craft text.While previous work has tackled this problem with models trained specifically to do the fill-in-theblank task, a more useful model is one that can effectively perform both FITB and continuation.In this work, we evaluate the feasibility of using a single model to do both tasks.We show that models pre-trained with a FITBstyle objective are capable of both tasks, while models pre-trained for continuation are not.Finally, we show how FITB models can be easily finetuned to allow for fine-grained control over the length and word choice of the generation.