Text content and task performance in the evaluation of a Natural Language Generation system
Albert Gatt, François Portet, International Conference RANLP - 2009 /Recent Advances in Natural Language Processing · OAR@UM (University of Malta) · 2009
An important question in the evaluation of Natural Language Generation systems concerns the relationship between textual characteristics and task performance. If the results of task-based evaluation can be correlated to properties of the text, there are better prospects for improving the system. The present paper investigates this relationship by focusing on the outcomes of a task-based evaluation of a system that generates summaries of patient data, attempting to correlate these with the results of an analysis of the system’s texts, compared to a set of gold standard human-authored summaries.