Server-Heartbeat-Based Task Failure Prediction: An Industrial Case Study
Mestan Fırat Çeliktuğ, Huseyin Gomleksizoglu · 2024
In this paper, we address the crucial need for effective and efficient Task Failure Prediction in Service Orchestration and Automation Platforms (SOAP).These complex systems, with inherent dependencies, are prone to task failures; which leads to service disruptions and increased costs. To tackle this challenge, we present a robust Task Failure Prediction Framework designed explicitly for SOAP environments. This framework leverages server heartbeats as a key data source for predicting task failures. It adopts a dual prediction approach, offering users the flexibility to choose between the INIT method (a rapid prediction method based on the initial server state) and the EXECUTION method (a comprehensive approach utilizing complete historical resource metrics).This paper makes several significant contributions, including offering a new perspective on the relationship between system resource features and task failure prediction; providing a systematic end-to-end solution for mitigating task failure risks in the SOAP context, including adaptable data collection, analysis, model training, and prediction for both online and offline scenarios; optimizing task failure prediction for specific SOAP use cases through "Task-wise," "Server-wise," and "Task Type-wise" strategies; and contributing to the understanding of task failure prediction and service orchestration automation by addressing some of the existing challenges and gaps in the literature.We confirm the framework’s validity in predicting task failures based on empirical results from use cases. Specifically, in a case study involving two distinct tasks—UCS-1 (a web service-based task) and UCS-2 (a memory-intensive task)—the EXECUTION method, leveraging Random Forest and XGBoost algorithms, achieved a perfect accuracy 1.0000 for predicting failures in UCS-1, with respective accuracies of 0.9580, and 0.9630 for UCS-2. This underscores the power of using a "Task-wise" complete resource usage history for precise failure prediction.We also highlight the inherent trade-off between accuracy and resource efficiency. While the EXECUTION method excels in accuracy, the INIT method offers substantial efficiency gains while maintaining a commendable level of accuracy. This flexibility empowers SOAP users to tailor the framework to their specific requirements and needs.We make our framework’s current implementation and dataset excerpts available on GitHub to support future research efforts1.