Big data platforms as a service: challenges and approach
James L. Horey, Edmon Begoli, Raghul Gunasekaran, Seung-Hwan Lim, James J. Nutaro · 2012
Infrastructure-as-a-Service has revolutionized the man-ner in which users commission computing infrastruc-ture. Coupled with Big Data platforms (Hadoop, Cassan-dra), IaaS has democratized the ability to store and pro-cess massive datasets. For users that need to customize or create new Big Data stacks, however, readily avail-able solutions do not yet exist. Users must first acquire the necessary cloud computing infrastructure, and man-ually install the prerequisite software. For complex dis-tributed services this can be a daunting challenge. To ad-dress this issue, we argue that distributed services should be viewed as a single application consisting of virtual machines. Users should no longer be concerned about individual machines or their internal organization. To illustrate this concept, we introduce Cloud-Get, a dis-tributed package manager that enables the simple instal-lation of distributed services in a cloud computing en-vironment. Cloud-Get enables users to instantiate and modify distributed services, including Big Data services, using simple commands. Cloud-Get also simplifies cre-ating new distributed services via standardized package definitions. 1