A versatile data-intensive computing platform for information retrieval from big geospatial data
Pierre Soille, Abigail Burger, Davide De Marchi, Pieter Kempeneers, Daniel Rodríguez, Vasileios Syrris, V. E. Vasilev · Future Generation Computer Systems · 2017
The increasing amount of free and open geospatial data of interest to major societal questions calls for the development of innovative data-intensive computing platforms for the efficient and effective extraction of information from these data. This paper proposes a versatile petabyte-scale platform based on commodity hardware and equipped with open-source software for the operating system, the distributed file system, and the task scheduler for batch processing as well as the containerization of user specific applications. Interactive visualization and processing based on deferred processing are also proposed. The versatility of the proposed platform is illustrated with a series of applications together with their performance metrics.