Evaluation of abstracts based on surface information of sentences
Yuji Ichioka, Tsuyoshi Yamamura, Yuji Sagawa, Noboru Ohnishi, Noboru Sugie · Systems and Computers in Japan · 2000
The abstract of a paper has a significant role in that it helps the reader decide whether or not to read the paper. This paper proposes a method to evaluate whether or not the abstract contains sentences that outline the contents of the paper (such as methods, experiments, and results). As the first step, the concept of the semantic role of the sentence is introduced, and the classification of the roles is defined. Then, 77 abstracts published in transaction journals are examined, and the relation between the surface information of the sentence and the semantic role is analyzed. Based on the result, a method that tries to estimate the semantic role is proposed. Applying the method to 511 sentences in 77 papers, the results for 413 sentences (81%) agreed with the evaluator's estimation of the semantic roles. In addition, the evaluation criterion that the already known information should not occupy more than half of the abstract, while the conditions and results of the author's own effort should be included, is tested. In this case, 8 of 50 abstracts are flagged as unsatisfactory. © 2000 Scripta Technica, Syst Comp Jpn, 31(3): 104–114, 2000