Software Artifacts for Performance Feedback Autoscaling Experiments with Workloads of Workflows in Apache Airflow
Alexey Ilyushkin, André Bauer, Alessandro V. Papadopoulos, Ewa Deelman, Alexandru Iosup · Zenodo (CERN European Organization for Nuclear Research) · 2019
These software artifacts are related to the computational artifacts DOI:10.5281/zenodo.2635573 The software artifacts are packed in the sa.tar.gz archive with the following directories: airflow contains the source code of the Airflow system v1.9.0 with the added autoscaling support, including the source code of all the considered autoscaling policies: PLF, SCF, and PFA. The code of the autoscalers is located in `airflow/airflow/autoscalers`. gurobi contains our implementation of the MIP model used in the paper for the Gurobi solver v8.0.1. tools contains the scripts we used to generate workloads, analyze the results, plot data, and deploy and run the Airflow experimental setup. For the experimental purposes the Airflow code can be just run directly without installation (e.g., without using the `airflow/INSTALL`, `airflow/setup.py` files). And that is how we were running it. The airflow code from the `airflow` directory should be correctly positioned, so that the deployments scripts can reach it. Check the paths inside the scripts within `tools/deployment`, and correct them, if necessary.