Microservices Architecture for Scalable Real-Time Data Processing at the Edge
Varun Kumar Chowdary Gorantla · International Journal of Emerging Trends in Computer Science and Information Technology · 2024
Based on the emerging fields of edge computing and IoT that provide increased scalability, low latency, and tolerance to failures in terms of data processing. It cannot meet such demand levels, especially through traditional monolithic architectures, since these are plagued with high latencies arising from processing bottlenecks and centralized architectures that are not scalable. In this paper, architecture for processing raw data in real-time at the edge using microservices architecture is proposed to improve the system’s efficiency and scalability with the help of the containerization technique, orchestration, and event-driven publishing and listening into the system. Splitting the processing, storage, and communication services into individual microservices makes the different services independent and makes adaptability, modularity, and scalability possible. Docker, Kubernetes, gRPC, MQTT, and Apache Kafka are some technologies used for Mesh and easy deployment of edge computing from node to node. Various tests prove that latency is decreased by an empty-nesting 70-90% and resource performance is increased by 40% less cloud reliance in opposition to a regular monolithic framework based in clouds. Some of the critical issues are network instabilities and fluctuations, security threats and concerns regarding their access, and managing scarcity; the future of AI-based orchestrations, federated learning, and 5 G-based edge computing is also explored. The conclusions presented in the paper would indicate that the future of real-time application in industries can be transformed with the help of microservices-based edge architectures in industrial automation, smart cities, health care, and self-sufficient systems. Drawing lessons from this work enhances the knowledge of context-aware scalable and robust edge computing and advances the area of microservices at the edge in practice