Research on Self-learning of Information Combination in Web Collecting
Lin Zhang · Computer Technology and Development · 2013
In order to precisely obtain Web pages on the topic,the Web crawler usually uses various Web information to forecast the linkages' value. In this paper,in order to improve the Web crawlers' accuracy in forecasting linkages' value,a Web searching strategy is proposed,which can automatically adjust the importance of various Web information according to the crawled Web pages. This crawler has learning ability,which can analyze the importance of Web information through crawling the training set,and then adjust the weights of Web information,get a better search strategy corresponding to actual Web. The algorithm and traditional Web information combination algorithm is compared. The experiment result shows that compared with the Web crawler based on fixed weights of Web information,the new crawler has higher searching accuracy.