SemEval-2015 Task 5: QA TempEval - Evaluating Temporal Information Understanding with Question Answering
Héctor Llorens, Nathanael Chambers, Naushad UzZaman, Nasrin Mostafazadeh, James F. Allen, James D. Pustejovsky · 2015
QA TempEval shifts the goal of previous TempEvals away from an intrinsic evaluation methodology toward a more extrinsic goal of question answering.This evaluation requires systems to capture temporal information relevant to perform an end-user task, as opposed to corpus-based evaluation where all temporal information is equally important.Evaluation results show that the best automated TimeML annotations reach over 30% recall on questions with 'yes' answer and about 50% on easier questions with 'no' answers.Features that helped achieve better results are event coreference and a time expression reasoner.