Feature-based object identification for web automation
Christoph Herzog, Iraklis Kordomatis, Wolfgang Holzinger, Ruslan R. Fayzrakhmanov, Bernhard Krüpl-Sypien · 2013
In this paper, we address automatic identification of common functional structures on web pages, a fundamental problem for web automation applications and graphical user interface testing. In contrast to other approaches, we aim to identify relevant patterns without relying on the source code of a web page or keywords, utilizing mostly geometrical and visually perceptible properties. We achieve this by transforming pages into an independent geometrical representation, on top of which we extract a set of features that allows us to employ traditional machine learning techniques for the identification task. We evaluate this approach by analyzing three typical scenarios, reviewing the obtained information retrieval key metrics and estimating the relevance of the chosen features. Our initial results demonstrate the feasibility of the proposed approach.