Evaluating the performance of the OntoSem semantic analyzer
Sergei Nirenburg, Stephen Beale, Marjorie J. McShane · 2004
This paper describes an innovative evaluation regimen developed for the text meaning representations (TMRs) produced by the Ontological Semantic (OntoSem) general purpose syntactic-semantic analyzer. The goal of evaluation is not only to determine the quality of TMRs for given texts, but also to assign blame for various classes of errors, thus suggesting directions for continued work on both knowledge resources and processors. The paper includes descriptions of the OntoSem processing environment, the evaluation regime itself and results from our first evaluation effort.