CoDa: Constrained Generation based Data Augmentation for Low-Resource NLP
Chandra Kiran Reddy Evuru, Sreyan Ghosh, Sonal Kumar, S Ramaneswaran, Utkarsh Tyagi, Dinesh Manocha · 2024
We present CoDa (Constrained Generation based Data Augmentation), a controllable, effective, and training-free data augmentation technique for low-resource (data-scarce) NLP.Our approach is based on prompting off-theshelf instruction-following Large Language Models (LLMs) for generating text that satisfies a set of constraints.Precisely, we extract a set of simple constraints from every instance in the low-resource dataset and verbalize them to prompt an LLM to generate novel and diverse training instances.Our findings reveal that synthetic data that follows simple constraints in the downstream dataset act as highly effective augmentations, and CoDa can achieve this without intricate decoding-time constrained generation techniques or fine-tuning with complex algorithms that eventually make the model biased toward the small number of training instances.Additionally, CoDa is the first framework that provides users explicit control over the augmentation generation process, thereby also allowing easy adaptation to several domains.We demonstrate the effectiveness of CoDa across 11 datasets spanning 3 tasks and 3 low-resource settings.CoDa outperforms all our baselines, qualitatively and quantitatively, with improvements of 0.12%-7.19%.Code is available 1 .1 https://github.com/Sreyan88/CoDa* These authors contributed equally to this work.Method Original 1: Scott Gimple sets the record straight on that big midseason finale scene.Original 2: mine is all of Isaiah chapter 6 because that is what Jesus did for me. 1.The document should have the following keywords: Grindr, terminate, determined or concluded, repeat or habitual, but should not have the following keywords: suspend, ascertained.2. The document should be potentially unfair.Here are also some examples: {exemplar 1},{exemplar 2},{exemplar 3} 3. The document should have parts-of-speech sequence similar to PROPN AUX ADV VERB ADP NOUN DET NOUN ADP NOUN PRON AUX VERB ADP PROPN PART AUX PUNCT VERB NOUN PUNCT PUNCT.4. The document should have a length of 19-28 words.5. Any sentence in the document should not include the abstract concept control.6.Any sentence in the document should not include the abstract concept use.7. Any sentence in the document should not include the abstract concept governance.