Focussed Crawling of Environmental Web Resources: A Pilot Study on the Combination of Multimedia Evidence
Theodora Tsikrika, Anastasia Moumtzidou, Stefanos Vrochidis, Ioannis Yiannis Kompatsiaris · 2014
This work investigates the use of focussed crawling tech-niques for the discovery of environmental multimedia Web resources that provide air quality measurements and fore-casts. Focussed crawlers automatically navigate the hyper-linked structure of the Web and select the hyperlinks to follow by estimating their relevance to a given topic, based on evidence obtained from the already downloaded pages. Given that air quality measurements and particularly air quality forecasts are presented not only in textual form, but are most commonly encoded as multimedia, mainly in the form of heatmaps, we propose the combination of textual and visual evidence for predicting the benefit of fetching an unvisited Web resource. First, text classification is applied to select the relevant hyperlinks based on their anchor text, a surrounding text window, and URL terms. Further hy-perlinks are selected by combining their text classification score with an image classification score that indicates the presence of heatmaps in their source page. A pilot evalu-ation indicates that the combination of textual and visual evidence results in improvements in the crawling precision over the use of textual features alone.