Research on web topic detection based on domain lexicon
Zhao Zhibin, Jia Yanfeng, Bao Yubin · 2013
Web topic detection is a crucial prerequisite to web-based data integration and also a key component for Vertical Search Engine. So, it attracts much attention from not only the industry but also the literature. In this paper, we proposed a domain-lexicon-based framework for Web topic detection. In our framework, we extracted the topical features from the web page first. Next, we employed Vector Space Model(VSM) and Support Vector Machine (SVM) to compute the topical relevance between the Web page features and the domain that the user prefers so as to conclude whether the web page satisfies the user's request. Vector Space Model is suitable for the domains where the corresponding domain lexicons need to be updated frequently. Oppositely, Support Vector Machine is suitable for the domains where the corresponding domain lexicons are relatively unchangeable. Moreover, in this work we also explored the mechanism of domain lexicon updating, which can guarantee the accuracy and freshness of the domain lexicon. Finally, we conducted extensive experiment to test our framework and analyze how the domain lexicon affects the judgement result.