Not so creepy crawler
Franziska von dem Bussche, Klara Weiand, Benedikt Linse, Tim Furche, François Bry · 2010
Web crawlers are increasingly used for focused tasks such as the extraction of data from Wikipedia or the analysis of social networks like last.fm. In these cases, pages are far more uniformly structured than in the general Web and thus crawlers can use the structure of Web pages for more precise data extraction and more expressive analysis.