CRAWLING OF JAPANESE REAL-ESTATE WEBSITES USING SCRAPY
Bassam Farooq, Mohd. Shahid Husain, Mohammad Suaib · International Journal of Advanced Research in Computer Science · 2018
Web crawler is a program in the softwarespace that enables the download of data from websites.This paper implements a python web crawler framework,Scrapy.The crawler framework implemented mainlyfocusses on major real-estate websites of Japan. Themotivation behindthe implementation of the Scrapyframework was the speed of website crawling supplied by the framework of Scrapy, data filters that can beappliedand also, the wide library support for pythonprogramming language.