An Adaptive Partitioning Method for Distributed Stream Processing Systems
Kazuma Tomatsu, Haruka Tanaka, Daisuke Kasamatsu · 2022 IEEE 11th Global Conference on Consumer Electronics (GCCE) · 2022
In recent years, stream processing systems have attracted attention for their ability to process big data, that is constantly generated, with low latency. However, the performance of stream processing deteriorates due to the bias in the amount of input data for parallel processing. This paper proposes an adaptive partitioning method based on the latency of each parallel process. Experimental results using a prototype system show that the latency bias can be reduced to half.