Research of web information mining by using crawler techniques
Qingcheng Li, Shan Lin, Zhenhua Dong · 2008
As the Internet rapidly becomes one of the most important information medium, web information mining has been the focus of several recent research projects and papers. There are massive documents in certain formats on the Internet while web crawlers building up with millions of computers scratch the Internet pages every second. Why not combine these two efficiently? This paper describe a new thought that mining web information by using crawler techniques. After explain the basic principle of crawler techniques, we present the architecture of the new web information mining system. For the initial test, the system is applied to mine certain standard formatted documents; the experimental data is reported in section IV. By the analysis of the result, we can approve that the system shows high efficiency, flexibility and low cost by using crawler techniques.