Web Honeypots for Spies
Blake Henderson, Sean Mckenna, Neil C. Rowe · 2018
We are building honeypots for document-collecting spies who are searching the Web for intelligence information. The goal is to develop tools for assessing the relative degree of interest elicited by users in representative documents. One experiment set up a site with bait documents and used two site-monitoring tools, Google Analytics and AWStats, to analyze the traffic. Much of this traffic was automated ("bots"), and showed some interesting differences in the retrieval frequency of documents. We also analyzed bot traffic on a similar real site, the library site at our school. In nearly one million requests, we concluded 64% were bots. 46 did identify themselves as bots, 40 came from blacklisted sites, and 12 gave demonstrably false user identifications. Requestors appeared to prefer documents to other types of files and 40% of the requests did not respect the terms of service on access provided by a robots. txt file.