Web robot detection algorithm based on episode analysis
Weigang Guo · Jisuanji gongcheng · 2005
After analyzing the navigational patterns of Web robots, the feature was found that the access sequence of robots usually didn't satisfy the hyperlink relations. The concept of episode was defined and a new algorithm based on episode analysis was proposed. The experiments show that the new algorithm can detect the unknown robots and unfriendly robots who do not obey the standard for robot exclusion.