A critical review of system-centred to user-centred evaluation of automatic abstracting research
Frances Johnson · e-space (Manchester Metropolitan University) · 1999
This paper presents a critical review of the evaluation objectives and techniques employed in automatic abstracting systems research. Various regimes have been adopted to provide some measure on which to estimate the merits of automatic abstracting techniques, yet few indicate the functionality of the document summary representations produced. As such the question of how best to generate abstracts to effectively convey the message of a text remains unanswered. Comparisons are drawn with the evaluation of information retrieval systems to highlight the inherent limitations of previous approaches to evaluation. This perspective for the critical analysis leads to the recommendation for a user centred approach to evaluation in the context of interactive retrieval systems and tasks.