Optimizing Load Distribution in Big Data Ecosystems
R.P. Diwakar, Rahul Sharma, Deviprasad Nayak, Hitesh Mohapatra · Advances in computational intelligence and robotics book series · 2025
In today's digital world, managing large volumes of data, known as “Big Data,” presents a significant challenge due to its volume and complexity. Regular software often struggles to handle this, necessitating the use of Load Balancing—a crucial aspect of cloud computing. Load balancing distributes workloads across resources, preventing slowdowns, reducing processing time, and optimizing system performance. This paper explores load balancing strategies in big data processing, including Round Robin, Least Connection, Resource-Based, Task-Based, and Dynamic methods, discussing their pros and cons. Effective load balancing ensures optimal resource usage, higher scalability, increased availability, dependability, fault tolerance, and improved performance. The paper provides a literature review, proposes a model for optimal load balancing, and tests it in a simulated environment, highlighting key findings and suggesting future research directions.