Consistent regions
Gabriela Jacques-Silva, Zheng Fang, Daniel Debrunner, Kun‐Lung Wu, Victor Dogaru, Eric N. Johnson, Michael W. Spicer, Ahmet Erdem Sarıyüce · Proceedings of the VLDB Endowment · 2016
Guaranteed tuple processing has become critically important for many streaming applications. This paper describes how we enabled IBM Streams, an enterprise-grade stream processing system, to provide data processing guarantees. Our solution goes from language-level abstractions to a runtime protocol. As a result, with a couple of simple annotations at the source code level, IBM Streams developers can define consistent regions , allowing any subgraph of their streaming application to achieve guaranteed tuple processing. At runtime, a consistent region periodically executes a variation of the Chandy-Lamport snapshot algorithm to establish a consistent global state for that region. The coupling of consistent states with data replay enables guaranteed tuple processing.