Automating Continuous Architecting for Big Data
Ramakrishna Ramadugu · 2025
As big data applications become increasingly complex, their architectures-comprising various middleware and processing components-demand continuous refinement to maintain optimal performance. This paper presents OSTIA (On-the-fly Static Topology Inference Analysis), a tool designed to support the continuous architecting of dataintensive applications (DIAs) by addressing three key areas: detecting common anti-patterns, performing algorithmic manipulations, and facilitating formal verification of architectural models. OSTIA operates by reverse-engineering the topologies of big data applications, such as those built on Apache Storm and Apache Hadoop, to ensure compliance with framework-specific constraints and identify potential design flaws. Through case studies involving both industrial and open-source applications, I demonstrate OSTIA's effectiveness in improving the reliability and performance of streaming and batch-processing systems. This paper also highlights OSTIA's recent enhancements, including formal verification capabilities and additional heuristics, and discusses its impact on the iterative process of continuous architecting.