Detecting Multi-Word Expressions Improves Word Sense Disambiguation
Mark Alan Finlayson, Nidhi Kulkarni · 2011
Multi-Word Expressions (MWEs) are prevalent in text and are also, on average, less polysemous than mono-words. This suggests that accurate MWE detection should lead to a nontrivial improvement in Word Sense Disambiguation (WSD). We show that a straightforward MWE detection strategy, due to Arranz et al. (2005), can increase a WSD algorithm’s baseline f-measure by 5 percentage points. Our measurements are consistent with Arranz’s, and our study goes further by using a portion of the Semcor corpus containing 12,449 MWEs- over 30 times more than the approximately 400 used by Arranz. We also show that perfect MWE detection over Semcor only nets a total 6 percentage point increase in WSD f-measure; therefore there is little room for improvement over the results presented here. We provide our MWE detection algorithms, along with a general detection framework, in a free, open-source Java library called jMWE. Multi-word expressions (MWEs) are prevalent in text. This is important for the classic task of Word Sense Disambiguation (WSD) (Agirre and Edmonds, 2007), in which an algorithm attempts to assign to each word in a text the appropriate entry from a sense inventory. A WSD algorithm that cannot correctly detect the MWEs that are listed in its sense inventory will not only miss those sense assignments, it will also spuriously assign senses to MWE constituents that themselves have sense entries, dealing a double-blow to WSD performance. Beyond this penalty, MWEs listed in a sense inventory also present an opportunity to WSD algorithms- they are, on average, less polysemous than mono-words. In Wordnet 1.6, multi-words have an average polysemy of 1.07, versus 1.53 for monowords. As a concrete example, consider sentence She broke the world record. In Wordnet 1.6 the lemma world has nine different senses and record has fourteen, while the MWE world record has only one. If a WSD algorithm correctly detects MWEs, it can dramatically reduce the number of possible senses for such sentences.