A First Evaluation of Logic Form Identification Systems
Vasile Rus · Meeting of the Association for Computational Linguistics · 2004
This paper presents a first experience with evaluating sytems that address the issue of Logic Form Identification (LFi). A Gold Standard approach was used in which experts provide solutions to test data. The expert solutions, the gold standard, are then compared against outputs from participanting systems and different metrics observed. We proposed a few novel metrics, including precision and recall, that are further used to provide comparative results. The test data included 4155 arguments and 2398 predicates grouped in 300 sentences.