Word frequencies in scientific prose
Hans Paijmans · Unisa Institutional Repository (University of South Africa) · 1995
An overview is given of popular term-weighting schemes as used in Information Retrieval. The question is raised whether the word frequency based t f.idf weights may be used as a yardstick to compare schemes that are not based on frequency-characteristics, such as the first mention theory or schemes based on word categories. It is demonstrated that word categories and frequency derived weights have a close correlation but that weighting according to the first mentioned theory shows no agreement with frequency-based weights.