Replication Package: Context-Specific Automated Quality Assessment for System Models in MBSE
Konstantin Rupert Blaschke, Andreea Diana Folea, Michael Unterkalmsteiner, Javier González‐Huerta, Simon Barner · Zenodo (CERN European Organization for Nuclear Research) · 2026
Replication package for the paper "Context-Specific Automated Quality Assessment for System Models in MBSE" (CoSAMoQA), accepted at the International Conference on the Quality of Information and Communications Technology (QUATIC 2026). The package contains everything needed to reproduce the empirical results of the paper: the (pseudonymized) data extracted from two versioned AutoFOCUS3 system-model repositories, the structured annotation guideline with its Model Quality Requirement (MQR) definitions, a pre-trained comprehensibility estimator, and a set of Jupyter notebooks. Run in order, the notebooks reproduce the inter-rater agreement analysis across the five quality goals, the comprehensibility-estimator training pipeline (feature selection and Random Forest with time-series-aware validation), the paper figures, and the labeling-speed analysis. Privacy note: personal identifiers in the data are pseudonymized — committer email addresses, commit-author identities, and commit message text are removed or replaced with neutral values, and the two labelers in the inter-rater study are referred to as "Reviewer A" and "Reviewer B". All metric values, timestamps, hashes, and numeric counts are unchanged.