Design and implementation of focused crawler algorithms of product reviews
Yan Shi · Journal of Jilin University · 2012
From the product reviews extracting algorithm,focused crawler algorithms of product reviews were designed based on the general crawler principle and the OpenAPI respectively,using OpenAPI call and link tracking method.Then the comments′ collection procedure of taobao.com and 360buy.com were realized.Finally both algorithms are compared with the general crawler algorithm,and some merits who are targeted program,timeliness data collection and easily embedded development were demonstrated.The results provide a good idea for the research on real-time reviews data collecting technology.