The Challenge of using Map-reduce to Query Open Data
Mauro Pelucchi, Giuseppe Psaila, Maurizio Toccu · 2017
For transparency and democracy reasons, a few years ago Public Administrations started publishing data sets concerning public services and territories. These data sets are called open, because they are publicly available through many web sites. Due to the rapid growth of open data corpora, both in terms of number of corpora and in terms of open data sets available in each single corpus, the need for a centralized query engine arises, able to select single data items from within a mess of heterogeneous open data sets. We gave a first answer to this need in (Pelucchi et al., 2017), where we defined a technique for blindly querying a corpus of open data. In this paper, we face the challenge of implementing this technique on top of the Map-Reduce approach, the most famous solution to parallelize computational tasks in the Big Data world.