SemEval-2010 Task 17: All-words Word Sense Disambiguation on a Specific Domain
Eneko Agirre, Oier López de Lacalle, Christiane Fellbaum, Andrea Marchetti, Antonio Toral, Piek T. J. M. Vossen, Mari Ostendorf, Michael Collins, Shri Narayanan, Douglas W. Oard, Lucy Vanderwende · 2009
Domain portability and adaptation of NLP components and Word Sense Disambiguation systems present new challenges. The difficulties found by supervised systems to adapt might change the way we assess the strengths and weaknesses of supervised and knowledge-based WSD systems. Unfortunately, all existing evaluation datasets for specific domains are lexical-sample corpora. This task presented all-words datasets on the environment domain for WSD in four languages (Chinese, Dutch, English, Italian). 11 teams participated, with supervised and knowledge-based systems, mainly in the English dataset. The results show that in all languages the participants where able to beat the most frequent sense heuristic as estimated from general corpora. The most successful approaches used some sort of supervision in the form of hand-tagged examples from the domain