Cloud-Agnostic Serverless Platform for Fault-Tolerant Execution of Dynamic Task Graphs
Quentin Delamea, Janna Burman, Jérôme Gurhem, Mohamed Khairallah, Wilfried Kirschenmann, Florian Lemaitre, Stéphane Vialle · 2025
Serverless computing simplifies cloud development by abstracting away infrastructure complexity. However, most serverless platforms lack the expressiveness required for complex and dynamic workloads. They also face challenges in achieving fault tolerance and often rely on proprietary cloud services. This paper introduces ArmoniK, an open-source, cloud-agnostic serverless platform for executing dynamic, DAG-based workloads. Its core features include subtasking for runtime graph reshaping, fault-tolerant scheduling, and adaptive resource management. ArmoniK runs at scale on any cloud or on-premises infrastructure. Evaluations show that it efficiently handles millions of tasks, scales seamlessly to thousands of workers, and remains reliable even under severe failure conditions, such as widespread instance preemptions. Already deployed in production, ArmoniK brings the benefits of serverless computing to demanding, business-critical applications.