The automatic generation of literature abstracts: an approach based on the identification of self-indicating phrases
Chris D. Paice · 1980
Considering the important part played by abstracts in the traditional information services, the possibility of producing abstracts by computer has not received very much attention. There are perhaps two main reasons for this. First, it appears tha t the production of well-constructed abstracts is an artificial intelligence problem, and therefore unlikely to be either feasible or worthwhile until well into the future: the alternative of picking sentences from here and there in a document is a rather unattractive proposition. Second, the cost of key-punching complete texts for input to an abstracting program can hardly be justified especially since the program will then in effect discard most of the text which has been so laboriously prepared. It now appears that the first of these objections is exaggerated --reasonable-looking abstracts can often be produced by quite 'unintelligent' programs -whi le with advances in technology the second problem should soon disappear. We should be ready to take advantage of this when it happens. Early work in this field concentrated on the extracting problem: that is to say, with finding sentences which could be extracted from a text to convey a good idea of its subject matter. Luhn (1958) write a program which looked for sentences containing clusters of 'key words': that is, the most frequent noncommonplace words in the text. The clusters were weighted according to their size and density, and those sentences containing the most highly weighted clusters were selected. At about the same time Baxendale (1958) drew attention to the fact that the position of a sentence within a text has a bearing on its importance: for instance, she showed that in 85 per cent of a sample of 200 paragraphs the 'topic' sentence was the first, while in another 7 per cent it was the last. Extending this idea, we can understand that the first few and the last few paragraphs of a document are likely to give a strong indication of its overall subject: the pages in between usually contain a lot of detail, which is not of much value taken out of its context. During the 1960s the most important work was carried out by Edmundson (1969), who studied four extracting methods, both individually and in all possible combinations. All four methods involved the assignment of weights to sentences, and the subsequent selection of sentences with the highest weights. The location method weighted sentences if they occurred in preferred positions