Ensuring Optimal Performance and Resilience for the Kafka Platform in A Hybrid Environment
Ramasankar Molleti · International Journal of Intelligent Systems and Applications in Engineering · 2024
Apache Kafka, an open-source distributed streaming platform, is now a critical part of any modern data architecture, particularly in the hybrid setup. This paper aims to discuss best practices that can be employed regarding Kafka to make it more performant and robust in on-premise or Cloud hybrid environments. As this research compares theoretical strategies regarding cluster configuration, networks’ optimization, data partition, fault tolerance, disaster recovery, and security, it offers a practical reference for practitioners. Also, there are examples of industries to know how they have implemented the respective applications and what practices are considered effective. The discussion is wrapped up by the evaluation of present problems and possible development for further research, which is beneficial to both academics and practitioners. DOI: https://doi.org/10.17762/ijisae.v12i1.6723