From chocolate bunny to chocolate crocodile: Do Language Models Understand Noun Compounds?
Albert Coil, Vered Shwartz · 2023
Noun compound interpretation is the task of expressing a noun compound (e.g.chocolate bunny) in a free-text paraphrase that makes the relationship between the constituent nouns explicit (e.g.bunny-shaped chocolate).We propose modifications to the data and evaluation setup of the standard task (Hendrickx et al., 2013), and show that GPT-3 solves it almost perfectly.We then investigate the task of noun compound conceptualization, i.e. paraphrasing a novel or rare noun compound.E.g., chocolate crocodile is a crocodile-shaped chocolate.This task requires creativity, commonsense, and the ability to generalize knowledge about similar concepts.While GPT-3's performance is not perfect, it is better than that of humans-likely thanks to its access to vast amounts of knowledge, and because conceptual processing is effortful for people (Connell and Lynott, 2012).Finally, we estimate the extent to which GPT-3 is reasoning about the world vs. parroting its training data.We find that the outputs from GPT-3 often have significant overlap with a large web corpus, but that the parroting strategy is less beneficial for novel noun compounds.