Apache Wayang in Action: Enabling Data Systems Integration via a Unified Data Analytics Framework
Kaustubh Beedkar, Aurélien Bertrand, Haralampos Gavriilidis, Augusto J. M. da Fonseca, Zoi Kaoudi, Mingxi Liu, Volker Markl, Juri Petersen, Fabio Andre Machado Porto, Victor de Paula Dornellas Ribeiro, Mads Sejer Pedersen, Lara Giovanna de Andrade Tavares, Michalis Vargiamis, Chen Xu · 2025
Apache Wayang is an open-source framework, which provides a systematic and efficient solution for unifying data analytics over disparate data sources and via integrating multiple heterogeneous data systems. It achieves that by decoupling applications from the underlying systems. In addition, it provides an optimizer so that users do not have to specify the platforms on which their pipeline should run but the optimizer can determine the best way given a cost metric. In this demonstration, we showcase how the flexible architecture of Wayang enables seamless integration with multiple heterogeneous data systems and how the query optimizer can lead to better performance.