Topic-Driven Crawlers: Machine Learning Issues
Filippo Menczer, Gautam Pant, Padmini Srinivasan · 2002
Topic driven crawlers are increasingly seen as a way to address the scalability limitations of universal search engines, by distributing the crawling process across users, queries, or even client computers.