A Comparative Study of Parallel and Distributed Big Data Programming Models: Methodologies, Challenges, and Future Directions
Muhammad Wasim · Lahore Garrison University Research Journal of Computer Science and Information Technology · 2023
According to a survey conducted in 2021, users share about 4 petabytes of data on Facebook daily. The exponential increase in data (called big data) plays a vital role in machine learning, the Internet of Things (IoT), and business intelligence applications. Due to the rapid increase in big data, research in big data programming models gained much interest in the past decade. Today, many programming paradigms exist to handle big data, and selecting an appropriate model for a project iscritical for its success. This study analyzes big data programming models such as MapReduce, Directed Acyclic Graph (DAG), Message Passing Interface (MPI), Bulk Synchronous Parallel (BSP), and SQL. We conduct a comparative study of distributed and parallel big data programming models and categorize these models into three classes: traditional data processing, graph-based processing, and query-based processing models. Furthermore, we evaluate these programming models based on theirperformance, data processing, storage, fault-tolerant, suitable language, and machine learning support. We highlight the benchmarks with their characteristics used for big data programming models. Finally, we discuss the models' challenges and suggest future directions for the research community.