Research on Web Crawler Technology
Yu Hongbo · Journal of Dongguan University of Technology · 2011
As an indispensable part of search engine,web crawler is used to download web pages from Internet.Classifications and principles of web crawler are introduced.The topic crawler is designed in detail,and then technical problems of designing high-performance web crawler to be solved are summarized.