Hadoop in the browser: Simulating MapReduce in the browser
H. Bilen, Melissa de Zwart · Research Repository (Delft University of Technology) · 2016
With the current increase of user generated data, the need for tools to process large quantities of data is increasing. One of the courses of the Computer Science BSc curriculum is the Big Data Processing course. The Big Data Processing course teaches students ways of doing so. A popular and teached method is using MapReduce, a programming model to process large quantities of data. Big Data Processing students currently write their implementations for MapReduce related assignments of the lab in the Cloudera Virtual Machine (VM). This VM is slow, cannot be used interactively and it cannot be used to teach all MapReduce principles like memory separation. Since there are no existing solutions that solves these problems we have decided to write our own. Writing our own solution required diverse knowledge including but not limited to JavaScript, ECMAScript 6, WebWorkers, HTML5 (&CSS), PHP, MySQL, API development, Wordpress, Linux, QUnit and Selenium. Our solution, Trifle, overcomes previously mentioned and other shortcomings. Trifle is a web-based solution that simulates MapReduce within the browser. The framework uses JavaScript together with WebWorkers and our front-end is written using Wordpress. By simulating MapReduce within the browser we managed to create a solution that requires no cluster, is easy to use, works multi-platform and most importantly enables lecturers to teach some MapReduce principles that could not be taught before. Furthermore, we integrated a submissions system that can be used to do interactive lectures in which the lecturer can see problems in real-time and explain obstacles. A user test we have done verifies that Trifle helps to better grasp the idea behind MapReduce.