Exploiting Hadoop Topology in Virtualized Environments
Rosangela de Fátima Pereira Marquesone, Walter Akio Goya, Jan-Erik Mångs, Azimeh Sefidcon · 2014
Virtualization is a key technique to make anenvironment easier to manage in terms of resource allocation.MapReduce is a programming model that provides anabstraction to perform distributed computation for large datasets. Hadoop is a well-known framework that offers an opensourceimplementation for this model. Combining Hadoop andvirtualization techniques in cloud-computing environments canunveil great potential, especially for big data context. However,running MapReduce jobs on virtual machines has indicatedperformance issues not solved yet. In this paper we presentand discuss three scenarios regarding Hadoop topology in acloud infrastructure. The first scenario proposes to allocateHadoop daemons in a fully virtualized environment, the secondscenario presents a hybrid environment, and the third scenariosuggests to virtualize only MapReduce daemons.We also reportresults from a series of tests allocating Hadoop daemons in afully virtualized environment. Results show that adding virtualmachines to the cluster causes an overhead, decreases theefficiency of CPU utilization, and shortens the time slots forthe MapReduce jobs.