Efficient Data Partitioning Algorithms for Distributed Storage Systems
Arjun Patel · International Journal of Emerging Research in Engineering and Technology · 2021
Distributed storage systems are essential for managing large-scale data in modern computing environments. These systems rely on efficient data partitioning algorithms to ensure data is evenly distributed, minimize data movement, and optimize query performance. This paper explores various data partitioning algorithms, their strengths, and limitations. We present a comprehensive review of existing techniques and propose a novel algorithm that improves partitioning efficiency and load balancing. The proposed algorithm is evaluated through extensive simulations and real-world experiments, demonstrating significant improvements in performance metrics such as query latency and data locality. The paper also discusses the implications of these findings for the design and implementation of distributed storage systems