Automatic text analysis based on transition phenomena of word occurrences
Miranda Lee Pao · Journal of the American Society for Information Science · 1978
Abstract A method of selecting index terms directly from a word frequency list is described. The original idea was suggested by Goffman who reasoned that the most content‐bearing words of a given text would occur at the transition region at which Zipf's First Law of words of high frequency of occurrences begins to take on properties of words of low frequency of occurrences. Word frequencies of two articles were analyzed. Results seem to indicate that the automated selection of index terms from a frequency list holds some promise for automatic indexing.