Design and implementation of user customized topic focused crawler
Min Yu-li · Jisuanji gongcheng yu sheji · 2015
The traditional focused crawler can not work without train sets of correspond topics.To make the focused crawler adapt to more topics,a clustering-based adaptive topic model was proposed,which helped the focused crawler to work with some url with the same topic.The topic vector was obtained by clustering the initial page,and correspond page was found out to update the topic vector,the url with the best-first strategy was ordered then.Based on the adaptive topic model,a user customized topic focused crawler was implemented.Finally,an experiment was executed.The results prove the focused crawler with the adaptive topic model performs well.