Word sense disambiguation of adjectives using probabilistic networks
Gerald Chao, Michael G. Dyer · 2000
In this paper, word sense disambiguation (WSD) accuracy achievable by a probabilistic classifier, using very minimal training sets, is investigated. We made the assumption that there are no tagged corpora available and identified what information, needed by an accurate WSD system, can and cannot be automatically obtained. The lesson learned can then be used to focus on what knowledge needs manual annotation. Our system, named Bayesian Hierarchical Disambiguator (BHD), uses the Internet, arguably the largest corpus in existence, to address the sparse data problem, and uses WordNet's hierarchy for semantic contextual features. In addition, Bayesian networks are automatically constructed to represent knowledge learned from training sets by modeling the selectional preference of adjectives. These networks are then applied to disambiguation by performing inferences on unseen adjective-noun pairs. We demonstrate that this system is able to disambiguate adjectives in unrestricted text at good initial accuracy rates without the need for tagged corpora. The learning and extensibility aspects of the model are also discussed, showing how tagged corpora and additional context can be incorporated easily to improve accuracy, and how this technique can be used to disambiguate other types of word pairs, such as verb-noun and adverb-verb pairs.