Coconut: Contextualized Commonsense Unified Transformers for Graph-Based Commonsense Augmentation of Language Models
Jun-Hyung Park, Mingyu Lee, Junho Kim, SangKeun Lee · 2024
In this paper, we introduce COCONUT to effectively guide the contextualization of structured commonsense knowledge based on large language models.COCONUT employs a contextualized knowledge prompting scheme to gather high-quality contextualization examples from a large language model.These examples are subsequently distilled into small language models to enhance their contextualization capability.Extensive evaluations show that CO-CONUT considerably improves commonsense reasoning performance across diverse benchmarks, models, and settings, exhibiting its flexibility and universality in generating contextualized commonsense knowledge.Notably, COCONUT consistently outperforms the stateof-the-art technique by an average of 5.8% 1 .Q. What do people use to absorb extra ink from a fountain pen? (A) shirt pocket (B) calligrapher's hand (C) inkwell (D) desk drawer (E) blotter Commonsense QA fountain pen shirt pocket