STORYEVAL: An Empirical Evaluation Framework for Narrative Generation.
Jonathan Rowe, Scott W. McQuiggan, Jennifer L. Robison, Derrick R. Marcey, James C. Lester · 2009
Research in intelligent narrative technologies has recently experienced a significant resurgence. As the field matures, devising principled evaluation methodologies will become increasingly important to ensure continued progress. Because of the complexities of narrative phenomena, as well as the inherent subjectivity of narrative experiences, effectively evaluating intelligent narrative technologies poses significant challenges. In this paper, we present STORYEVAL, an evaluation framework for empirically studying computational models of narrative generation. Drawing on evaluation methodologies from cognitive science, human-computer interaction, and natural language processing, as well as techniques that have begun to emerge in the narrative technologies community, STORYEVAL consists of four complementary tools for evaluating both interactive and non-interactive narrative generation: