Evaluation of an Automatic Text Abstraction System
Wendy Wang, Sumali Pin-Ngern Conlon, Brian J. Reithel · Journal of the Association for Information Systems · 2003
Information overload has been a serious problem to scholars, researchers, and students in academia.There is too much to read and too little time.A good abstract would reduce readers reading time by providing gist of the source.By reading the abstract, readers can decide if it is worthwhile to read the original.Therefore, time could be saved.Since most journal articles do not observe guidelines published by the American National Standards Institute (ANSI) on writing abstracts, they need to be modified by professional abstractors.The huge volume of journals has brought pressure on professional abstractors in various bibliographic abstracting services.A text abstraction system was developed to produce abstracts that meet the ANSI standards.To evaluate the system performance, this study compared its output to two other abstracts: the abstract written by the original author and the manual modification of the abstract system output.All three abstracts are derived from the same source paper.The evaluation results indicated that the author abstract differed significantly from the manually edited system output in terms of meeting the ANSI standards.The abstract was also significantly different from the system output both before and after manual editing in fluency.All three were not significantly different in understandability.No significant difference was identified between the system output and its manual edited version in any of the three dependent variables.Therefore, it is possible to use the system output directly without manual modification.This would, subsequently, save time and resources.