A Topic Crawler Algorithm Based on Semantic Analysis
Xu Zhao · Computer Engineering and Science · 2010
Massive web and its rapid growth make it difficult for general-purpose search engines to provide satisfactory results for the theme-or area-oriented queries. This paper studies the subject of gathering information relevant to the subject,to significantly reduce the amount of web pages dealing. By assessing the degree of Web pages,it gives priority to the crawling pages related to a higher degree. Using a subspace-based semantic analysis technique,combined with the Bayesian mechanism and support vector machine,we design and implement an efficient topic crawler. Experiments show that our algorithm has good accuracy and efficiency.