Macrocoding the Lexicon with Co-occurrence Knowledge
Frank Smadja · Psychology Press eBooks · 2021
Neither syntax nor semantics can justify the use of a certain class of English word combinations. This class contains word pairs that often appear together in a given context of meaning. Such pairs are called co-occurrence relations or idiosyncratic collocations . To correctly understand or produce natural language, such lexical relations need to be specifically encoded in the lexicon. In this work we focus on language generation, co-occurrence knowledge is particularly important for this task because without it, awkward or incorrect sentences could be produced. Most previous work either deferred the issue or handled it by hand encoding the knowledge. In this chapter, we show how word-based lexicons can be enriched with automatically acquired lexical relations. We call this process macrocoding the lexicon, because it corresponds to the pre-encoding of lexical associations in a regular lexicon. We describe Xtract , a co-occurrence compiler that retrieves lexical relations from the analysis of a large corpus and we show how this acquired knowledge can be used in the framework of language generation.