Research and Design of Web Crawler for Music Resources Finding
Xiu Xia Chen, Wen Qian Shang · Applied Mechanics and Materials · 2014
This paper designs an automatic web crawler system which crawls music resources on the Internet. Firstly, this paper gives the architecture of the system and the function of each module; then describes the detailed design of each module; Finally, the key technologies and algorithms used in the system are given in a detailed description, including the use of χ2 statistics to select feature words, TF-IDF algorithm to calculate the weights of feature words, the correlation of web page and music theme using vector space model.