A Statistical Model for Measuring Structural Similarity between Webpages
Zhenisbek Assylbekov, Assulan Nurkas, Inês Russinho Mouga · Recent Advances in Natural Language Processing · 2015
This paper presents a statistical model for measuring structural similarity between webpages from bilingual websites. Starting from basic assumptions we derive the model and propose an algorithm to estimate its parameters in unsupervised manner. Statistical approach appears to benefit the structural similarity measure: in the task of distinguishing parallel webpages from bilingual websites our languageindependent model demonstrates an Fscore of 0.94–0.99 which is comparable to the results of language-dependent methods involving content similarity measures.