A Document Clustering Algorithm for Web Search Engine Retrieval System
Hongwei Yang · 2010
As the number of available Web pages grows, it is become more difficult for users finding documents relevant to their interests. Clustering is the classification of a data set into subsets (clusters), so that the data in each subset (ideally) share some common trait - often proximity according to some defined distance measure. It can enable users to find the relevant documents more easily and also help users to form an understanding of the different facets of the query that have been provided for web search engine. A popular technique for clustering is based on K-means such that the data is partitioned into K clusters. In this method, the groups are identified by a set of points that are called the cluster centers. The data points belong to the cluster whose center is closest. Existing algorithms for k-means clustering are slow and do not scale to large number of data points and converge to different local minima based on the initializations. A fast greedy k-means algorithm can attack both these drawbacks, but it is a limitation when the algorithm is used for large number of data points, So we introduce an efficient method to compute the distortion for this algorithm. The experiment results show that the fast greedy algorithm is superior to other method and can help users to find the relevant documents more easily than by relevance ranking.