The collection and analysis of the data on job advertisements with the use of big data
Jacek Maślankowski · Wiadomości Statystyczne The Polish Statistician · 2019
The goal of this paper is to present, on the one hand, the benefits for offi-cial statistics (labour market) resulting from the use of web scraping methods to gather data on job advertisements from websites belonging to big data compilations, and on the other, the challenges connected to this process. The paper introduces the results of experimental research where web-scraping and text-mining methods were adopted. The analysis was based on the data from 2017–2018 obtained from the most popular job-searching websites, which was then collated with Statistics Poland’s data obtained from Z-05 forms. The above-mentioned analysis demonstrated that web-scraping methods canbe adopted by public statistics services to obtain statistical data from alternative sourcescomplementing the already-existing databases, providing the findings of such researchremain coherent with the results of the already-existing studies.