Implicit Parallelism through Deep Language Embedding
Alexander K. Alexandrov, Andreas Kunft, Asterios Katsifodimos, Felix Schuler, Lauritz Thamsen, Odej Kao, Tobias Herb, Volker Markl · 2015
The appeal of MapReduce has spawned a family of systems that implement or extend it. In order to enable parallel collection processing with User-Defined Functions (UDFs), these systems expose extensions of the MapReduce programming model as library-based dataflow APIs that are tightly coupled to their underlying runtime engine. Expressing data analysis algorithms with complex data and control flow structure using such APIs reveals a number of limitations that impede programmer's productivity.