From TimeLines to StoryLines: A preliminary proposal for evaluating narratives
Egoitz Laparra, Itziar Aldabe, Germán Rigau · 2015
We formulate a proposal that covers a new definition of StoryLines based on the shared data provided by the NewsStory workshop. We re-use the SemEval 2015 Task 4: Timelines dataset to provide a gold-standard dataset and an evaluation measure for evaluating StoryLines extrac-tion systems. We also present a system to explore the feasibility of capturing Story-Lines automatically. Finally, based on our initial findings, we also discuss some sim-ple changes that will improve the existing annotations to complete our initial Story-Line task proposal. 1