Research and Realization of Intelligent Focused Web Crawler
Cong Wang · 2006
An Intelligent Focused Web Crawler(IFWC) is investigated detailedly.Supported by comprehensive information theory,Concept-based VSM is applied to filter the crawled web pages,while extended metadata semantic relevance algorithm for predicting the relativity between URL and topic.The result of experiments has shown that IFWC has more accurate and wider coverage for web pages relevant to a predefined set of topics.