A New Approach to Pseudoword Generation
Lubomír Otrusina, Pavel Smrž · 2010
Sense-tagged corpora are used to evaluate word sense disambiguation (WSD) systems.Manual creation of such resources is often prohibitively expensive.That is why the concept of pseudowords -conflations of two or more unambiguous words -has been integrated into WSD evaluation experiments.This paper presents a new method of pseudoword generation which takes into account semanticrelatedness of the candidate words forming parts of the pseudowords to the particular senses of the word to be disambiguated.We compare the new approach to its alternatives and show that the results on pseudowords, that are more similar to real ambiguous words, better correspond to the actual results.Two techniques assessing the similarity are studied -the first one takes advantage of manually created dictionaries (wordnets), the second one builds on the automatically computed statistical data obtained from large corpora.Pros and cons of the two techniques are discussed and the results on a standard task are demonstrated.Even though the wordnet-based method improves the modelling accuracy of the generated pseudowords (and one of the automatic corpus-based procedures provides results close to it), the key observation is that the results of WSD on the Senseval-like data is highly unstable and that the comparison of the WSD systems should be application-specific.