Improvement of TF-IDF Algorithm Based on Hadoop Framework
Bin Li, Guoyong Yuan · 2012
TF-IDF algorithm is often used in search engine, text similarity computation, web data mining, etc.These applications are often faced with the massive data processing.Therefore, how to calculate the tf-idf quickly and efficiently is very important.In this paper, we give a tf-idf algorithm based on the hadoop framework.Experiments show that in the case of massive data computing, the new method applying hadoop framework is more efficient than the traditional methods.