Analysis of Parallel Optimisation Strategies Based on MapReduce Models
Shiyue Wu · International Journal of Computer Science and Information Technology · 2024
The aim of this paper is to provide an in-depth analysis of parallel analysis strategies for MapReduce models, and to explore how to improve the overall performance by optimising task allocation and scheduling, improving data locality and increasing node utilisation. The research methodology includes an analysis and overview of existing MapReduce frameworks and proposes a series of improvement strategies. These strategies improve the utilisation of computing resources by adjusting the granularity of task division, optimising data slicing and distribution, and improving task scheduling algorithms. The results show that by reasonably optimising the parallel analysis strategy of the MapReduce model, its performance in large-scale dataset processing can be significantly improved, especially in resource-constrained distributed environments. Ultimately, this paper concludes that although the MapReduce model has been used in more mature applications, there is still much room for optimising its parallel strategy when facing larger scale and complex data processing tasks. In the future, further research should be devoted to finer-grained task scheduling, dynamic resource allocation, and more efficient fault-tolerance mechanisms to continuously improve the parallel processing capability of the MapReduce model.