WebT-IDC: A Web Tool for Intelligent Dataset Creation A Use Case for Forums and Blogs
Boudabous Maroua, Anna Pappa · Procedia Computer Science · 2021
This paper presents WebT-IDC, an end-to-end Web Tool for Intelligent Dataset Creation, able to construct noiseless corpora of feedback on different topics in any language, from web forums and blogs. The method is based on a unique extraction model of pagination element pattern, independent of DOM structure. WebT-IDC is a holistic tool covering all stages, from user’s query, crawling and scraping web pages and extracting relevant data, to building text corpora, free from boilerplate and repetitive data, useful in machine learning tasks. The output is a partially labeled dataset, accurate to reflecting a realistic vision of the market, tested in a Transformer based model in order to show the relevance for immediate use in ML task, offering high accuracy in polarity, language and product recognition. The system is evaluated in terms of noise filtering efficiency and computing time as well as precision and recall.