Support for data-intensive computing with CloudMan
Yousef Kowsar, Enis Afgan · International Convention on Information and Communication Technology, Electronics and Microelectronics · 2013
Infrastructure-as-a-Service (IaaS) compute infrastructure model has showcased its ability to transform how access to compute resources is realized; it delivered on the notion of Infrastructure-as-Code and enabled a new wave of compute adaptability. In previous work we have been developing CloudMan (usecloudman.org) as a versatile solution for enabling and managing compute clusters in cloud environments via a simple web interface or an API. However, CloudMan only supported batch processing workloads. As the magnitude of the data produced and processed in digital form grows, the need to support big data applications in clusters in the cloud becomes more evident. In this paper, we have extended the batch processing capability of CloudMan and presenteda novel architecture for supporting big data analysis workloads in cluster-in-the-cloud environment. We also implemented the details through CloudMan using established big data platforms.