Open-Source MLOps Configurator
To Ngoc Lam, Linh · Zenodo (CERN European Organization for Nuclear Research) · 2025
Early-stage machine learning teams frequently encounter significant challenges when implementing MLOps practices, primarily due to fragmented tooling, manual workflows, and limited reproducibility. Despite the availability of numerous open-source frameworks, they often lack seamless integration, leaving practitioners with insufficient guidance on how to build coherent, maintainable toolchains. This research addresses this critical gap by developing an Open-Source MLOps Configurator to facilitate the transition from MLOps Level 0 to Level 1 using lightweight, community-driven tools. The configurator integrates a structured questionnaire with a multi-criteria decision analysis (MCDA) framework. User inputs, reflecting priorities, constraints, and specific workflow characteristics, are translated into criterion targets within the MCDA model. Each tool in the catalogue is assessed against analytically defined functional and adoption-oriented criteria across six MLOps domains: experiment tracking, data versioning, object storage, model registry, data validation, and model deployment. Recognizing that many early-stage users may be unfamiliar with the MLOps tool landscape, the configurator not only recommends a suitable toolchain but also clarifies how each tool aligns with the user's distinct requirements and how the selected components can interoperate effectively within a cohesive Level-1 workflow. Based on the final recommendations, the system generates reproducible project artifacts, including documentation, configuration files, and template scripts for training and evaluation. These outputs illustrate how Level-1 practices such as structured environments, versioned artifacts, and modular execution can be consistently implemented without necessitating enterprise-scale infrastructure. The validation process emphasized analytical correctness rather than user testing. The system was verified to produce consistent rankings under fixed inputs, uphold internal coherence between the questionnaire logic, scoring rules, and tool metadata, and generate syntactically valid project artifacts. Although the tool catalogue and scoring formula represent initial versions that require ongoing refinement, the results indicate that a structured, questionnaire-driven approach can significantly enhance the accessibility of MLOps practices for teams embarking on their maturity journey. This work provides a transparent, extensible framework for evaluating open-source MLOps tools, along with a proof-of-concept configurator that effectively bridges theoretical maturity models with practical, reproducible project scaffolding within the Level-1 MLOps framework.