Adaptive topology decomposition for storm
Cheng Xie, Liang Qian, Lianghui Ding, Yang Feng · 2017
Real-time computing system attracts more and more attention in both academic researches and industrial applications. One of the real-time computing systems, Apache Storm, because of its characteristics of stream processing and high fault tolerance, is widely used for machine learning and distributed remote process call (RPC), etc. However, the existing approaches to decompose topology for Storm cannot ensure an optimized performance. In this paper, we propose an adaptive topology decomposition algorithm for Storm where topology decomposition based on cluster status and components of topology can be performed at run time. We have evaluated the processing performance and the load balancing of the algorithm. The evaluation results indicate that the proposed algorithm has better performances on task processing and load-balancing than the existing algorithms.