A Framework to define optimized Shuffling process to reduce data transfer latency: MapReduce in Serverless Infrastructure
Achyut Anantakumar Vadavadagi · TRAP@NCI (National College of Ireland) · 2019
Today in cloud computing, serverless platform is appeared as a modern technology with the combination of Function as a Service (FaaS) and Backend as a Service (BaaS) with significant computational power. The FaaS is the class of serverless computing service in which user has a privilege to deploy their functions with no management of hardware's by the user. It enable the users to run multiple tasks concurrently with higher elasticity and scalability by introducing fine-grained billing. Cloud providers adds resources arbitrary to deploy and run functions by millisecond level billing. MapReduce is framework or model for decentralized processing of data, where it is broadly operates for processing the large data-sets. This paper considers MapReduce in serverless platform where shuffling creates the latency issues because the object doesn't scale due to Input Output per second limitations during processing of data-sets. This paper provides the Composite Model for processing large datasets defined in greater performance serverless platform by executing the tasks on Function as a Service (AWS Lambda) and Backend as a service (bringing combination of fast storage and slow storage). An evaluation has been carried out to understand the suitability of MapReduce in serverless platform and experimentation is carried out and compare with other platforms. The results shows that AWS lambda is suitable for processing the large data-sets with less execution time with moderate billing.