Efficient web crawling with proposed URL ordering
Sandhya Pundhir, Mohammad Qasim Rafiq, Omar Farooq · 2011
In this paper an efficient and modified ordering algorithm is proposed. Which shows the better utilization of storage space. This paper shows the results of the modified proposed URL (Uniform Resource Locator) ordering algorithm. Experimental result shows that it can work efficiently in finding the important pages comparatively to the traditional PageRank. We have performed experiment on data set using web logs from university and we achieve statistical significant improvement in ordering by getting two more categories on an average also more URLs in the higher order rank. And it shows 6.95% time saving in downloading and 46.6% better in Quality than traditional PageRank.